System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting contents of a presentation by a user; means for automatically generating presentation materials based on the inputted contents; means for displaying the generated presentation materials; means for analyzing a moving image of a presentation practice and providing feedback based on a result of the analysis; means for performing a model presentation with a synthesized voice similar to a voice of the user; and means for practicing the presentation in a scene set using virtual reality.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For people who are not good at creating presentation materials or who have little experience, creating presentation materials takes a lot of time, making it difficult to prepare effective presentations. To solve this problem, a quick way to create presentation materials and an effective way to practice is required, but conventional methods have not been able to fully achieve this. In particular, there is a lack of an environment for receiving effective feedback or a means to provide realistic presentation practice simulations. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for a user to input presentation content, a means for automatically generating presentation materials based on the input content, a means for displaying the generated presentation materials, a means for analyzing a video of a practice presentation and providing feedback based on the results, a means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, and a means for practicing the presentation in a scene set using virtual reality.This system allows users to quickly create presentation materials and receive specific feedback based on the analysis.Furthermore, practicing using virtual reality allows users to practice in an environment that is similar to a real scene, thereby improving their presentation skills.

[0006] "User" means a person who uses the system to create and present presentation materials.

[0007] A "presentation" is a method of conveying information to an audience about a specific topic or content.

[0008] "Content" refers to the subject matter, points, and information you want to convey in your presentation.

[0009] "Materials" are slides or documents that visually or writtenly represent the content of the presentation.

[0010] A "generative AI model" is an artificial intelligence algorithm that automatically creates presentation materials based on data entered by the user.

[0011] "Analysis" refers to data analysis processes such as audio analysis and image analysis that are carried out to evaluate the presentation practice videos.

[0012] "Feedback" is information that indicates the evaluation results of the presentation practice and areas for improvement.

[0013] "Synthetic voice" is an artificial voice that a program generates to resemble the user's voice.

[0014] "Demonstration" means reproducing the content of the presentation as a model using synthesized voice.

[0015] "Virtual reality" refers to a virtual environment that uses computer technology to recreate a realistic experience.

[0016] A "simulation" is a virtual exercise that recreates a real-life presentation environment in virtual reality for practice. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The system of the present invention is designed to assist users who are not good at or have little experience creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[0039] Document creation function

[0040] User Action:

[0041] The user enters the content of the presentation into the designated input form. For example, if the user wants to present about the market launch plan of a new product, the user writes the content in detail in the input form. Once the input is complete, the user clicks the "Submit" button.

[0042] The device:

[0043] The input presentation content is sent to the server, where it is properly formatted.

[0044] The server:

[0045] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. For example, it generates a title slide, product features slide, market analysis slide, introduction schedule slide, and conclusion slide based on the input topic, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the device.

[0046] The device:

[0047] The received presentation materials are displayed to the user, who can then review them and make corrections or additions as necessary.

[0048] Evaluation function

[0049] User Action:

[0050] Users record videos of their practice presentations and upload them to the system.

[0051] The device:

[0052] The recorded video file is sent to the server.

[0053] The server:

[0054] The server that receives the video performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, specific evaluations are made such as "speaking too fast" or "unfocused gaze," and feedback is generated based on the results. This feedback is then sent to the device.

[0055] The device:

[0056] The received feedback is displayed to the user, allowing them to see areas for improvement and correction, and to adjust their next practice based on this.

[0057] Demonstration function

[0058] User Action:

[0059] Users upload the presentation materials they have created and a sample of their own voice (audio file) to the system.

[0060] The device:

[0061] Send the uploaded data to the server.

[0062] The server:

[0063] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice samples. The server then uses the voice to demonstrate the presentation materials and generate a synthetic voice presentation. The generated audio file is then sent to the device.

[0064] The device:

[0065] The demonstration audio is played for users, who can use it as a model to improve their presentation content and speaking style.

[0066] Virtual practice function for each scene

[0067] User Action:

[0068] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[0069] The device:

[0070] Send scene setting information to the server.

[0071] The server:

[0072] The server receives the scene information and generates the specified virtual environment, and the virtual environment data is sent to the device and the VR goggles.

[0073] The device:

[0074] The received VR environment data is transferred to the VR goggles.

[0075] User Action:

[0076] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual large hall, they can gain experience in a situation similar to a real presentation.

[0077] As described above, the system of the present invention provides consistent support from the rapid creation of presentation materials to effective practice methods, thereby improving the presentation skills of users.

[0078] The processing flow will be explained below.

[0079] Document creation function

[0080] Processing Steps:

[0081] Step 1:

[0082] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[0083] Step 2:

[0084] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[0085] Step 3:

[0086] The device formats the presentation content it receives and sends it to the server.

[0087] Step 4:

[0088] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[0089] Step 5:

[0090] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[0091] Step 6:

[0092] Save the generated presentation materials in PDF or PPT format and send them to your device.

[0093] Step 7:

[0094] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[0095] Evaluation function

[0096] Processing Steps:

[0097] Step 1:

[0098] A user records a video of their practice presentation, which includes the user giving the presentation.

[0099] Step 2:

[0100] Users upload recorded videos to the system.

[0101] Step 3:

[0102] The device sends the uploaded video file to the server.

[0103] Step 4:

[0104] The server analyzes the received video, specifically, analyzing the audio and images.

[0105] Step 5:

[0106] The server extracts evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it evaluates the speaker's speaking speed and eye movement as "fast" or "unfocused."

[0107] Step 6:

[0108] The server generates feedback based on the analysis results, including suggestions for improvement and correction.

[0109] Step 7:

[0110] The server generates feedback and sends it to the device.

[0111] Step 8:

[0112] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[0113] Demonstration function

[0114] Processing Steps:

[0115] Step 1:

[0116] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[0117] Step 2:

[0118] The device sends the uploaded presentation materials and voice sample to the server.

[0119] Step 3:

[0120] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[0121] Step 4:

[0122] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[0123] Step 5:

[0124] The server transmits the generated demonstration audio file to the terminal.

[0125] Step 6:

[0126] The device displays a demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[0127] Virtual practice function for each scene

[0128] Processing Steps:

[0129] Step 1:

[0130] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[0131] Step 2:

[0132] The device sends the scene setting information to the server.

[0133] Step 3:

[0134] The server generates a specified virtual environment based on the scene information received.

[0135] Step 4:

[0136] The VR environment data generated by the server is sent to the terminal and VR goggles.

[0137] Step 5:

[0138] The terminal transfers the received VR environment data to the VR goggles.

[0139] Step 6:

[0140] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual hall, they can gain experience in a situation that is close to a real presentation.

[0141] Example 1

[0142] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Creating and delivering presentations is a difficult, time-consuming and laborious task for many users, placing a significant burden on beginners and those with little experience. This issue extends beyond support for creating presentation materials to the improvement of actual presentation skills. Conventional methods require time to create presentation materials and lack efficient practice methods, making it difficult to effectively improve presentation skills in a short period of time.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0145] In this invention, the server includes means for analyzing the presentation content and automatically generating presentation materials using a generative AI model, means for saving the generated presentation materials in PDF or PPT format and sending them to a terminal, and means for analyzing a presentation practice video and providing feedback based on the results. This allows for the rapid generation of presentation materials based on the content entered by the user, and further analysis of the practice video and provision of feedback, enabling effective improvement of presentation skills in a short period of time.

[0146] "User" refers to a person who uses the system to create or practice presentation materials.

[0147] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[0148] "Server" refers to the computer system that processes data, runs the generative AI model, stores and transmits generated materials, and provides analysis and feedback.

[0149] A "generative AI model" refers to an artificial intelligence model that automatically generates presentation materials based on input text data.

[0150] "Presentation materials" refers to slides, documents, etc. that visually display the contents of the presentation.

[0151] "Input form" refers to the interface that allows users to input the content of the presentation.

[0152] "Feedback" refers to evaluations and suggestions for improvement provided to users that are generated based on the results of analyzing the presentation practice video.

[0153] "Synthetic voice" refers to artificial voice data generated to resemble the user's voice.

[0154] "Virtual reality" refers to a simulated environment generated using computer technology.

[0155] "Simulation" refers to the near-realistic reproduction of a particular scenario or environment.

[0156] "Prompts" refer to guidelines or instructions used by generative AI models to generate presentation materials.

[0157] The system of the present invention is designed to assist users who are not good at creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[0158] First, the user enters the content of the presentation into a specified input form. For example, if a user wants to present a new product's market launch plan, they enter the details and click the "Submit" button. The device then converts the input presentation content into an appropriate format and sends it to the server using a protocol such as an HTTP POST request.

[0159] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (e.g., OpenAI's GPT-4). Specifically, based on the input theme, it generates a title slide, product features, market analysis, introduction schedule, and conclusion slides, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the terminal using file transfer protocols (FTP or HTTP).

[0160] The device displays the received presentation materials to the user, who can then review them and make corrections or additions as necessary.

[0161] Example prompt sentence:

[0162] "Please create a presentation about your new product launch plan. Key points should include product features, market analysis, launch schedule, and expected results."

[0163] Next, the user can record a video of the practice presentation and upload it to the system. The device sends the recorded video file to the server using a protocol such as an HTTP POST request.

[0164] After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the user's speaking speed or their eye gaze is not focused, and generate detailed feedback based on the results. The generated feedback is sent to the device in a text message format (e.g., JSON).

[0165] The device displays the received feedback to the user, allowing the user to see areas for improvement or correction.

[0166] In addition, users can upload their created presentation materials and a sample of their own voice (audio file) to the system. The device sends the data to the server, which uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice sample. The presentation materials are then demonstrated using this synthetic voice, and a synthetic voice presentation is generated. The generated presentation is then sent to the device.

[0167] The device plays back the received synthesized speech presentation, allowing the user to use it as a model to improve their presentation content and speaking style.

[0168] Finally, the user can select the VR practice mode and set the desired scene (e.g., a large hall or a conference room). The device sends the scene setting information to the server, which then generates the specified virtual environment. This virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine) and sent to the device and VR goggles.

[0169] The device then transfers the received VR environment data to the VR goggles, where the user can practice their presentation in the virtual environment. For example, by practicing a presentation in a virtual large hall, users can gain experience in a situation similar to a real presentation.

[0170] As described above, the system of the present invention consistently supports everything from the rapid creation of presentation materials to effective practice methods, thereby improving the user's presentation skills.

[0171] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0172] Step 1:

[0173] The user enters the details of the presentation in the designated input form. When the input is complete, the user clicks the "Send" button. The input is sent to the terminal as text data.

[0174] Step 2:

[0175] The terminal receives the text data entered by the user and formats it into an appropriate format (such as JSON). The formatted data is then sent to the server using an HTTP POST request. Specifically, the text data entered in the input form is converted into JSON format.

[0176] Step 3:

[0177] The server receives the JSON-formatted text data sent via an HTTP POST request, parses the received data, and extracts the theme and content of the presentation. During this analysis process, the analysis engine identifies keywords and context in the input data.

[0178] Step 4:

[0179] The server inputs the analyzed data into a generative AI model (e.g., GPT-4). The generative AI model automatically generates presentation materials based on the input data. The generative AI model generates slides containing content such as a title slide, product features, market analysis, introduction schedule, and conclusion.

[0180] Step 5:

[0181] The server saves the generated presentation slides in PDF or PPT format, and the saved files are sent to the terminal via HTTP or FTP. Specifically, the generated slides are converted to PDF format and saved in the specified directory.

[0182] Step 6:

[0183] The terminal receives the PDF or PPT file from the server and displays it to the user using a PDF reader or PPT viewer. The user can then review the document and make corrections or additions as necessary.

[0184] Step 7:

[0185] A user records a video of their practice presentation and uploads it to the system. The recorded video file is recognized by the device and sent to the server. Specifically, a video file is selected and uploaded to the server via an HTTP POST request.

[0186] Step 8:

[0187] The server receives the uploaded video file and performs audio and image analysis, extracting evaluation points such as speaking style, tone of voice, speed, eye movements, and posture.

[0188] Step 9:

[0189] The server generates feedback based on the analyzed evaluation points. The feedback is generated in text message format (e.g., JSON) and includes specific improvements and corrections. This feedback is sent to the device as an HTTP response.

[0190] Step 10:

[0191] The device analyzes the feedback received from the server and displays it to the user, who can then review the feedback and be aware of areas for improvement in their next practice session.

[0192] Step 11:

[0193] The user uploads the presentation materials they have created and a sample of their own voice (an audio file) to the system. The device sends the data to the server. Specifically, the presentation materials and audio file are selected and uploaded to the server via an HTTP POST request.

[0194] Step 12:

[0195] The server analyzes the received presentation materials and voice samples and uses a generative AI model to generate a synthetic voice that matches the content of the presentation materials.

[0196] Step 13:

[0197] The server then integrates the synthesized speech into the presentation materials to generate a live audio presentation, which is then saved as an audio file and sent to the device via HTTP or FTP.

[0198] Step 14:

[0199] The terminal plays the presentation with audio received from the server and displays it to the user, who can use it as a model to improve their own presentation skills.

[0200] Step 15:

[0201] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The device sends the setting information to the server. Specifically, the VR scene selection information is sent to the server in JSON format.

[0202] Step 16:

[0203] The server generates a virtual environment based on the VR scene information. The virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine).

[0204] Step 17:

[0205] The virtual environment data is sent from the server to the device and the VR goggles. The device then transfers the received VR environment data to the VR goggles. The user can then wear the VR goggles and practice their presentation in the virtual environment.

[0206] The above is the flow of program processing, including the specific operations performed at each step and an explanation of the inputs and outputs.

[0207] (Application example 1)

[0208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0209] Creating presentation materials and practicing presentations is difficult and time-consuming for many users. Furthermore, in brick-and-mortar stores, staff are required to be able to give efficient presentations, such as when selling new products or introducing services, and the associated skills training is also important. Therefore, there is a need for a system that allows users to efficiently create presentation materials, practice, evaluate, and demonstrate them, and also improve their presentation skills in brick-and-mortar stores.

[0210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0211] In this invention, the server includes a means for a user to input the content of a presentation, a means for automatically generating presentation materials based on the input content, a means for displaying the generated presentation materials, a means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, a means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and a means for generating a virtual environment within a physical store and practicing the presentation within that environment. This enables users to quickly create, evaluate, and receive feedback on their presentation materials, as well as provide effective training within the physical store.

[0212] "User" refers to an individual or organizational member who creates, evaluates, and practices presentation materials.

[0213] "Input means" refers to an interface or device that allows a user to provide specific content to a system.

[0214] "Automatic generation means" refers to software and algorithms for generating presentation materials based on specified inputs.

[0215] "Displaying means" refers to a display device or software that allows users to see the generated presentation materials and feedback.

[0216] "Video analysis means" means software and hardware for analyzing and evaluating the video and audio of a recorded presentation.

[0217] "Means for providing feedback" refers to interfaces and algorithms for providing users with suggestions for improvement and advice based on the analysis results.

[0218] "Means for demonstrating using synthetic speech" refers to technology that generates similar synthetic speech based on a user's voice sample and uses the speech sample to demonstrate the speech using presentation materials.

[0219] "Virtual reality methods" refers to the use of VR devices and software to create a virtual environment in which to practice a presentation.

[0220] "System" refers to the entire software and hardware collection that combines the above means to consistently support the creation, practice, and evaluation of presentation materials.

[0221] "Synthetic voice" refers to an artificial voice that is generated to resemble the user's voice.

[0222] "Virtual environment" refers to a computer-generated simulation that allows users to experience situations similar to those in the real world.

[0223] An "in-store" virtual environment refers to a virtual reality scene that mimics an actual in-store situation.

[0224] "Voice analysis" refers to the technology of analyzing recorded audio data to extract evaluation points such as tone of voice and speaking speed.

[0225] "Image analysis" refers to the technology of analyzing recorded video data to extract evaluation points such as eye movements and posture.

[0226] The system based on this invention basically comprises the following means: means for a user to input the content of the presentation, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and means for generating a virtual environment within a physical store and practicing the presentation in that environment.

[0227] User operations

[0228] First, the user logs in to the terminal and enters the content of the presentation into the designated input form. For example, if they are describing a market launch plan for a new product, they enter the details. Then, by clicking the "Submit" button, the input content is sent to the server.

[0229] Creating documents using generative AI models

[0230] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. For example, based on input about a new product's market launch plan, it generates a title slide, product features, market analysis, launch schedule, and conclusion slides. The materials are saved in PDF or PPT format and sent to the device.

[0231] View and modify materials

[0232] The terminal displays the generated material to the user, who can check it and make corrections or additions as necessary.

[0233] Presentation skills evaluation

[0234] Next, the user practices the presentation, records the video, and uploads it to the system. The device then sends the recorded video file to the server. After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. Specifically, evaluations are made on points such as "speaking too fast" and "unfocused eye contact," and feedback is generated based on the results. This feedback is sent to the device and displayed to the user. The user can use this information to improve their presentation skills.

[0235] Synthetic speech demonstration

[0236] Additionally, users upload their presentation materials and a sample of their own voice to the system. The server uses a generative AI model to generate a synthetic voice that resembles the user's voice. This voice is then used to create a demonstration voice using the presentation materials and sent to the device. This allows users to use the synthesized voice as a model to improve their presentation content and speaking style.

[0237] Virtual practice

[0238] The user selects the VR practice mode and sets the desired scene. For example, they can select a virtual environment such as a large hall, a conference room, or a brick-and-mortar store. The scene setting information is sent to the server, which then generates the specified virtual environment. The virtual environment data is then sent to the device and VR goggles. The user then puts on the VR goggles and practices their presentation in the virtual environment. This allows the user to gain experience in a situation that is close to an actual presentation scene.

[0239] Usage example

[0240] For example, if a store staff member were to prepare a presentation document that briefly explains the features of a new product, an "eco vacuum cleaner," they could use the system based on this invention. By inputting the following prompt sentence into the generative AI model, the document would be quickly created.

[0241] Example prompt sentence:

[0242] Create a sales presentation for an eco-friendly vacuum cleaner.

[0243] 1. Title Slide

[0244] 2. Product Features

[0245] 3. Market analysis

[0246] 4. Implementation Schedule

[0247] 5. Conclusion

[0248] In this way, by utilizing the system of the present invention, it is possible to quickly create presentation materials, practice and evaluate them effectively, and improve presentation skills in a physical store.

[0249] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0250] Step 1:

[0251] The user logs in to the terminal and enters the content of the presentation into the specified input form. The input form contains fields for describing details such as the presentation theme, purpose, and target audience, and the input is sent to the server by pressing the "Send" button. The input is in text format.

[0252] Step 2:

[0253] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. Specific prompts are passed to the generative AI model for analysis. For example, based on a new product's market launch plan, a title slide, product features, market analysis, launch schedule, and conclusions are generated. The generated slides are saved in PDF or PPT format and sent to the device.

[0254] Step 3:

[0255] The device displays the generated presentation materials to the user. The user can review the content based on these materials and make corrections or additions as necessary. The corrected data is also sent to the server, where it can be processed again by the generative AI model.

[0256] Step 4:

[0257] A user practices a presentation and records a video of it. Once the recording is complete, the user uploads the video file to the system. The device then sends the recorded video file to the server. The video file is in a common video format (e.g., MP4).

[0258] Step 5:

[0259] The server analyzes the received video file using audio and image analysis. Audio analysis evaluates voice tone and speaking speed, while image analysis evaluates eye movements and posture. These evaluation points are extracted and specific feedback (e.g., "You speak too fast" or "Your gaze is not focused") is generated. Based on this feedback, areas for improvement and correction are extracted and sent to the device.

[0260] Step 6:

[0261] The device displays feedback to the user, who can then take action to improve their presentation skills.

[0262] Step 7:

[0263] Users upload their created presentation materials and a sample of their own voice to the system. The server then uses a generative AI model to generate synthetic speech based on the user's voice sample. This synthetic speech is then used to demonstrate the presentation materials, providing the user with a model presentation to use.

[0264] Step 8:

[0265] The device plays a demonstration of the generated synthesized voice to the user, who can use this as a reference to further improve the content and speaking style of their presentation.

[0266] Step 9:

[0267] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, or a brick-and-mortar store). This scene information is sent to the server, which then generates the specified virtual environment.

[0268] Step 10:

[0269] The generated virtual environment data is sent to the VR goggles via the terminal. The user can then put on the VR goggles and practice their presentation in the virtual environment, allowing them to gain experience in a situation that is close to a real presentation.

[0270] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0271] The system of the present invention includes a means for a user to input the content of a presentation and automatically generate presentation materials using a generative AI model, a means for displaying the generated presentation materials, a means for analyzing a video of a practice presentation and providing feedback, a means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, a means for practicing the presentation in a scene set up using virtual reality, and a means for analyzing emotions by combining an emotion engine.

[0272] Document creation function

[0273] User Action:

[0274] The user enters the content of the presentation into a dedicated input form. For example, they can enter content such as "Marketing strategy for the next product." Once the input is complete, they click the "Submit" button.

[0275] The device:

[0276] The presentation content sent by the user is sent to the server. The input content is properly formatted.

[0277] The server:

[0278] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials, such as a title slide, marketing strategy details, market analysis, action plan, and conclusion slides.

[0279] The device:

[0280] The generated presentation materials are received in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[0281] Evaluation function

[0282] User Action:

[0283] Users record videos of their practice presentations and upload them to the system.

[0284] The device:

[0285] The recorded video file is sent to the server.

[0286] The server:

[0287] The received video is analyzed. Specifically, audio analysis, image analysis, and an emotion engine are used to analyze the user's emotions.

[0288] The server:

[0289] Based on the analysis results, the system evaluates the presentation's delivery style, tone of voice, speed, eye movements, posture, and even emotional aspects, and generates feedback, including specific suggestions for improvement such as "adjust your speaking speed" and "maintain a consistent gaze."

[0290] The device:

[0291] Feedback is received and displayed to the user, who can use it to improve their next practice.

[0292] Demonstration function

[0293] User Action:

[0294] Users upload their presentation materials and a sample of their voice.

[0295] The device:

[0296] These data are sent to the server.

[0297] The server:

[0298] Based on the presentation materials and audio samples, a generative AI model is used to generate a synthetic voice that resembles the user's voice. This synthetic voice is then used to generate a demonstration audio file using the presentation materials.

[0299] The device:

[0300] The system receives a demonstration audio file and plays it for the user, who can use it as a model to improve their presentation content and speaking style.

[0301] Virtual practice function for each scene

[0302] User Action:

[0303] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[0304] The device:

[0305] Send scene setting information to the server.

[0306] The server:

[0307] The specified virtual environment is generated based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[0308] The device:

[0309] The received VR environment data is transferred to the VR goggles.

[0310] User Action:

[0311] Users put on VR goggles and practice their presentations in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[0312] The role of the emotional engine

[0313] The server:

[0314] The server uses an emotion engine to collect emotional data from the user's facial expressions, tone of voice, and speaking style during presentation practice videos and real-time VR practice sessions, detecting emotions such as tension, joy, and impatience.

[0315] The server:

[0316] The emotion engine analyzes the detected emotional data and generates feedback based on it, including specific advice such as "You seem nervous, so try slowing down your speech" or "Speak more emotively."

[0317] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[0318] The processing flow will be explained below.

[0319] Document creation function

[0320] Processing Steps:

[0321] Step 1:

[0322] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[0323] Step 2:

[0324] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[0325] Step 3:

[0326] The device formats the presentation content it receives and sends it to the server.

[0327] Step 4:

[0328] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[0329] Step 5:

[0330] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[0331] Step 6:

[0332] Save the generated presentation materials in PDF or PPT format and send them to your device.

[0333] Step 7:

[0334] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[0335] Evaluation function

[0336] Processing Steps:

[0337] Step 1:

[0338] A user records a video of their practice presentation, which includes the user giving the presentation.

[0339] Step 2:

[0340] Users upload recorded videos to the system.

[0341] Step 3:

[0342] The device sends the uploaded video file to the server.

[0343] Step 4:

[0344] The server analyzes the received video. Specifically, it performs audio and image analysis, and then extracts emotional data using an emotion engine.

[0345] Step 5:

[0346] The server evaluates speaking style, tone of voice, speed, eye movements, posture, and emotions, and extracts evaluation points, such as "speaking too fast," "unfocused eye contact," and "appears nervous."

[0347] Step 6:

[0348] The server generates feedback based on the analysis results. The feedback includes suggestions for improving speaking style and posture, as well as emotional advice. For example, "You seem nervous, so try taking a deep breath" or "Try to lower your voice tone and speak calmly."

[0349] Step 7:

[0350] The server generates feedback and sends it to the device.

[0351] Step 8:

[0352] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[0353] Demonstration function

[0354] Processing Steps:

[0355] Step 1:

[0356] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[0357] Step 2:

[0358] The device sends the uploaded presentation materials and voice sample to the server.

[0359] Step 3:

[0360] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[0361] Step 4:

[0362] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[0363] Step 5:

[0364] The server transmits the generated demonstration audio file to the terminal.

[0365] Step 6:

[0366] The device plays the demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[0367] Virtual practice function for each scene

[0368] Processing Steps:

[0369] Step 1:

[0370] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[0371] Step 2:

[0372] The device sends the scene setting information to the server.

[0373] Step 3:

[0374] The server generates a specified virtual environment based on the scene information received.

[0375] Step 4:

[0376] The VR environment data generated by the server is sent to the terminal and VR goggles.

[0377] Step 5:

[0378] The terminal transfers the received VR environment data to the VR goggles.

[0379] Step 6:

[0380] The user puts on VR goggles and practices their presentation in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[0381] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[0382] Example 2

[0383] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] Traditional methods for creating, practicing, and evaluating presentation materials require a great deal of time and effort, making it difficult to improve the quality of presentations. In particular, in order to receive feedback on one's own performance, one must request evaluations from others, making it difficult to receive objective and continuous feedback. Furthermore, the content and structure of presentation materials must be created manually, making it difficult to generate materials quickly and effectively.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0386] In this invention, the server includes: means for a user to input presentation content; means for using a generative AI model to automatically generate presentation materials based on the input content; means for displaying the generated presentation materials; means for analyzing a video of a practice presentation and providing feedback based on the results using audio analysis, image analysis, and the generative AI model; means for demonstrating a model presentation using a synthesized voice that resembles the user's voice; means for practicing the presentation in a scene set using virtual reality; and means for analyzing emotional data during the presentation using an emotion engine and providing feedback. This allows users to quickly and automatically create presentation materials and receive multifaceted feedback to improve the quality of their presentations.

[0387] "Presentation content" refers to the information and data that a user inputs to create presentation materials.

[0388] A "generative AI model" is an artificial intelligence algorithm that automatically creates presentation materials based on input content.

[0389] "Presentation Materials" means the content of a presentation in slide or other format, typically generated as a PDF or PPT file.

[0390] "Voice analysis" is a technology that analyzes recorded voice data and evaluates the tone, speed, quality of the voice, etc.

[0391] "Image analysis" is a technology that analyzes recorded video data and evaluates eye movements, facial expressions, posture, etc.

[0392] "Feedback" is an evaluation of the content and performance of a presentation and specific advice for improvement.

[0393] "Synthetic voice" is an artificial voice that resembles the user's voice and is generated based on a user's voice sample.

[0394] "Virtual reality" refers to a virtual digital environment generated using computer technology in which users can interact.

[0395] The "emotion engine" is a technology that analyzes a user's emotions from facial expressions, tone of voice, etc., and detects specific emotional states.

[0396] A "scene" refers to a specific environment or location within virtual reality where a presentation takes place.

[0397] MODE FOR CARRYING OUT THE INVENTION

[0398] The system of the present invention automatically generates presentation materials based on the content entered by the user in order to improve the quality of presentations, and provides multifunctional support for supporting presentation practice.

[0399] Document creation function

[0400] User operations

[0401] The user enters the content of the presentation into a dedicated input form, for example, a specific content such as "Marketing strategy for the next product," and clicks the submit button.

[0402] Device operation

[0403] The device formats the input presentation content appropriately and sends it to the server. The data sent is in text format and is encoded.

[0404] Server Operations

[0405] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. Specifically, slides are generated, including a title slide, details of the marketing strategy, market analysis, an action plan, and a conclusion. Various natural language processing technologies are used in this generative AI model.

[0406] Device operation

[0407] The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[0408] Prompt Sentence Examples

[0409] "Please create a presentation about the marketing strategy for our next product."

[0410] Evaluation function

[0411] User operations

[0412] The user records a video of the presentation practice and uploads the video file to the system.

[0413] Device operation

[0414] The device sends the recorded video file to the server, where the file format and size are checked.

[0415] Server Operations

[0416] The server analyzes the received video using audio analysis, image analysis, and an emotion engine, and analyzes voice tone, speed, eye movement, posture, emotion, etc.

[0417] Server Operations

[0418] Feedback is generated based on the analysis results, including suggestions for improvement such as "adjust speaking speed" and "maintain consistent eye contact."

[0419] Device operation

[0420] The feedback is sent to the device and displayed to the user, who can use it to improve their next practice.

[0421] Prompt Sentence Examples

[0422] "Record yourself practicing your presentation and provide feedback."

[0423] Demonstration function

[0424] User operations

[0425] Users upload their own presentation materials and voice samples to the system.

[0426] Device operation

[0427] Your presentation and audio samples are sent to the server, where they are checked for correct file format and size.

[0428] Server Operations

[0429] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the uploaded voice sample, and then uses the generated synthetic voice to create a demonstration audio file of the presentation material.

[0430] Device operation

[0431] The demonstration audio file is sent to the terminal and played back for the user, who can use the audio file as a model to improve the content and speaking style of their presentation.

[0432] Prompt Sentence Examples

[0433] "Please convert your presentation materials into audio files and play them as a model."

[0434] Virtual practice function for each scene

[0435] User operations

[0436] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). Scenes can be selected from the system's catalog.

[0437] Device operation

[0438] Scene setting information is sent to the server, including the selected scene's ID and detailed parameters.

[0439] Server Operations

[0440] The server generates the specified virtual environment based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[0441] Device operation

[0442] The terminal transfers the received VR environment data to the VR goggles.

[0443] User operations

[0444] Users put on VR goggles and practice their presentation in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data.

[0445] Prompt Sentence Examples

[0446] "I want to practice my presentation in a virtual conference room environment."

[0447] By combining these functions, users can quickly create presentation materials and practice them with detailed feedback. Furthermore, by using an emotion engine, it is possible to analyze emotions such as tension and joy and provide more effective feedback.

[0448] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0449] Document creation function

[0450] Step 1:

[0451] User actions

[0452] The user enters the content of the presentation into a special input form and clicks the send button. The input is specific content such as "marketing strategy for the next product." The output is the content of the form being sent to the terminal as data.

[0453] Step 2:

[0454] Processing performed by the device

[0455] The device properly formats the presentation content received from the user and sends it to the server. The input is the text data of the presentation content sent by the user, and the output is sending the formatted data to the server. Specifically, the data is formatted, encoded, and sent to the server according to the communication protocol.

[0456] Step 3:

[0457] Processing performed by the server

[0458] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. The input is the formatted presentation content, and the output is the generated presentation material. Specific operations include text analysis, determining the slide structure and layout, and generating automatically inserted images and graphs.

[0459] Step 4:

[0460] Processing performed by the device

[0461] The terminal receives the generated presentation materials in PDF or PPT format and displays them to the user. The input is the presentation material file sent from the server, and the output is to display that file on the user's device. Specific operations include downloading the file, displaying it in a viewer, and accepting edits.

[0462] Evaluation function

[0463] Step 1:

[0464] User actions

[0465] A user records a video of a practice presentation and uploads the video file to the system. The input is the recorded video file, and the output is the uploaded video data.

[0466] Step 2:

[0467] Processing performed by the device

[0468] The device sends the recorded video file to the server. The input is the video file uploaded by the user, and the output is to send this file to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[0469] Step 3:

[0470] Processing performed by the server

[0471] The server uses audio analysis, image analysis, and an emotion engine to analyze the received video. The input is the received video file, and the output is the analysis results and feedback. Specific operations include analyzing voice tone and speed, analyzing eye movements and posture, and detecting emotional states.

[0472] Step 4:

[0473] Processing performed by the server

[0474] Feedback is generated based on the analysis results. The input is the analysis results, and the output is the generated feedback. Specific operations include extracting areas for improvement based on the analysis data and generating specific feedback statements.

[0475] Step 5:

[0476] Processing performed by the device

[0477] The terminal receives the feedback data and displays it to the user. The input is the feedback data sent from the server, and the output is to show this data to the user. Specific operations include displaying feedback text, hints, and operation guides for the user's next practice.

[0478] Demonstration function

[0479] Step 1:

[0480] User actions

[0481] Users upload their created presentation materials and their own voice samples to the system. The input is the presentation material file and the voice sample file, and the output is the transfer of these data to the terminal.

[0482] Step 2:

[0483] Processing performed by the device

[0484] The terminal sends presentation materials and audio samples to the server. The input is the file uploaded by the user, and the output is to send these files to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[0485] Step 3:

[0486] Processing performed by the server

[0487] The server uses a generative AI model to generate synthetic speech that resembles the user's voice based on the uploaded voice sample. This synthetic speech is then used to create a demonstration audio file of the presentation materials. The input is the voice sample and the presentation materials, and the output is the synthesized demonstration audio file. Specific operations include speech synthesis, audio file generation, and reading of the presentation materials.

[0488] Step 4:

[0489] Processing performed by the device

[0490] Receives a demonstration audio file and plays it for the user. The input is the audio file sent from the server, and the output is the playback of the audio file. Specific operations include downloading the file, playing it in an audio player, and accepting user operations.

[0491] Virtual practice function for each scene

[0492] Step 1:

[0493] User actions

[0494] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The input is the scene setting information, and the output is the set information being sent to the device.

[0495] Step 2:

[0496] Processing performed by the device

[0497] Sends scene setting information to the server. The input is the scene information set by the user, and the output is to send this information to the server. Specific operations include formatting the setting information and communicating it according to the transmission protocol.

[0498] Step 3:

[0499] Processing performed by the server

[0500] The server generates a virtual environment based on the received scene information. The input is the received scene information, and the output is the generated virtual environment data. Specific operations include 3D modeling of the virtual environment and scenario setting.

[0501] Step 4:

[0502] Processing performed by the device

[0503] The received VR environment data is transmitted to the VR goggles. The input is the virtual environment data sent from the server, and the output is the data transmitted to the VR goggles. The specific operations include decoding the data and transmitting it to the device.

[0504] Step 5:

[0505] User actions

[0506] Users put on VR goggles and practice their presentations in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data. The input is the actual presentation's movements and emotional data, and the output is real-time feedback.

[0507] The above are the processing steps in the system of the present invention, which allows users to achieve comprehensive improvement in their presentation skills.

[0508] (Application example 2)

[0509] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0510] Conventional presentation practice support systems incorporate functions for automatically generating presentation materials and providing feedback on practice, but they are not adequately suited to real-world work environments, particularly in factory environments. Furthermore, because they do not provide real-time feedback based on emotional data, it is difficult to effectively alleviate issues such as tension and impatience. This results in problems such as reduced user work efficiency and reduced teaching effectiveness.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0512] In this invention, the server includes means for a user to input presentation content, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for analyzing a video of a practice presentation and providing feedback based on the results, means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, means for practicing the presentation in a scene set using virtual reality, means for analyzing a recorded video of the practice presentation, collecting emotional data and providing feedback, and means for providing work guidance in a factory using virtual reality and robots. This allows users to practice in a simulation that resembles an actual work environment, and by receiving feedback based on the emotional data, the quality of their presentations can be improved.

[0513] "User" refers to a person who uses the presentation system to create presentation materials, practice them, and receive feedback.

[0514] "Presentation content" refers to the information and outline that a user enters when creating presentation materials.

[0515] "Auto-generation" refers to the process of automatically generating presentation materials using machine learning models and algorithms based on input information and prompts.

[0516] "Presentation materials" refers to materials such as slides and documents used to effectively communicate the content of a presentation.

[0517] "Display means" refers to a device or method for visually presenting the generated presentation materials to the user.

[0518] "Feedback" means information or comments that provide improvements or evaluations based on the results of your practice presentation.

[0519] "Synthetic voice" refers to a voice that resembles a human voice and is generated using voice synthesis technology.

[0520] "Virtual reality" means a virtual environment created using computer graphics and other technologies, in which users can have a realistic experience.

[0521] "Emotional data" refers to information about the user's emotional state analyzed from facial expressions, voice, etc.

[0522] "Work instruction" means the process of providing instruction and training regarding work tasks and procedures within a factory.

[0523] A "robot" refers to an automated machine that performs programmed actions and provides information and work instructions to users.

[0524] The system of the present invention allows users to input presentation content, automatically generate presentation materials using a generative AI model, display and evaluate the materials, and practice the presentation in a virtual reality environment. To effectively implement the system of the present invention, the following hardware and software are used:

[0525] Hardware and software used

[0526] 1. Hardware

[0527] Server: High-performance data processing server

[0528] Device: A computer or tablet that a user operates.

[0529] VR system: Virtual reality goggles such as Oculus Quest 2

[0530] Robot: A multi-functional robot that provides guidance and feedback within the factory

[0531] 2. Software

[0532] Generative AI model: OpenAI GPT-4

[0533] Sentiment analysis model: Custom trained model using TensorFlow and Keras

[0534] Video analysis software: OpenCV and MoviePy

[0535] Voice analysis software: Librosa

[0536] Data Transfer and Management System: Cloud Database

[0537] System processing overview

[0538] First, the user inputs the content of the presentation using a dedicated terminal. For example, they input a prompt such as "Operating procedure for new equipment." Once the input is complete, the terminal sends the content to the server.

[0539] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (OpenAI GPT-4). These materials include a title slide, operating procedures, safety precautions, etc. The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user.

[0540] Next, the user records a video of their practice presentation and uploads it to the server via their device. The server receives the video, performs audio analysis (Librosa) and image analysis (OpenCV), and extracts emotional data using emotion analysis models (TensorFlow and Keras). This allows the server to determine emotional states such as tension or joy, and generates feedback based on the results.

[0541] Furthermore, a synthesized voice generated from a sample of the user's voice is created to demonstrate the presentation materials and provided to the user, allowing the user to learn from the model presentation.

[0542] Additionally, using a virtual reality system (Oculus Quest 2), users can practice their presentations in a designated scene (e.g., a specific location in a factory). The server generates this virtual environment and provides real-time feedback based on emotional data.

[0543] Finally, the robot will provide work instructions within the factory, explaining specific operating procedures and providing real-time guidance based on presentation materials and feedback.

[0544] Specific examples

[0545] For example, when leading a presentation on new safety procedures in a factory, a worker inputs the prompt phrase "new safety procedure." The generative AI model automatically creates presentation materials and displays them on the device. The worker can then give a presentation based on these materials and automatically receive feedback from the recorded video.

[0546] Prompt Sentence Examples

[0547] Prompt: "Operating procedures for new equipment"

[0548] Generated presentation contents:

[0549] Title: New equipment operating procedures

[0550] Slide 1: Facility Overview

[0551] Slide 2: Operational Procedure 1

[0552] Slide 3: Operational Procedure 2

[0553] Slide 4: Safety Precautions

[0554] Slide 5: Questions and Answers

[0555] In this way, the system of the present invention can comprehensively improve users' presentation skills and work efficiency through the rapid creation of presentation materials, evaluation of practice, provision of advanced feedback using emotional data, and practical instruction using virtual reality and robots.

[0556] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0557] Step 1:

[0558] The user inputs the content of the presentation. The user enters a prompt sentence into a dedicated input form and clicks the submit button. For example, the user enters "Operating procedures for new equipment." This input becomes the initial data for the system.

[0559] Step 2:

[0560] The device sends the input presentation content to the server. The device then formats the input content and sends it to the server in a format that is easy for the generative AI model to analyze. This is where data is sent and received.

[0561] Step 3:

[0562] Based on the presentation content received by the server, a generative AI model (OpenAI GPT-4) is used to automatically generate presentation materials. The generative AI model analyzes the input prompts and creates slides (PDF or PPT format) to present relevant information, such as a title slide, operating procedures, and safety precautions.

[0563] Step 4:

[0564] The server sends the generated presentation materials to the terminal. The generated materials are transferred to the terminal in the appropriate format (PDF or PPT) and displayed to the user. The user can check the materials and make corrections as necessary.

[0565] Step 5:

[0566] The user records a video of their practice presentation and uploads it to the server via their device. The recorded video is sent to the server and becomes the data for the next analysis step.

[0567] Step 6:

[0568] The server analyzes the received video. Specifically, audio analysis (Librosa) and image analysis (OpenCV) are performed, and emotional data is extracted using an emotion analysis model (TensorFlow and Keras). For example, MFCC features are extracted to determine the emotional state. This allows emotions such as tension and joy to be detected.

[0569] Step 7:

[0570] The server generates feedback based on the analysis results, including adjustments to speaking speed and improvements to eye contact. The server creates the feedback information and notifies the device.

[0571] Step 8:

[0572] The device displays the received feedback to the user, who can review the feedback and receive guidance on how to improve the presentation's delivery and content.

[0573] Step 9:

[0574] The user uploads a sample of their voice to the server, which then serves as input for generating the synthesized voice.

[0575] Step 10:

[0576] The server generates synthetic speech. Using a generative AI model, it generates synthetic speech that resembles the user's voice. Based on this synthetic speech, a model presentation audio file is created.

[0577] Step 11:

[0578] The server sends the synthesized voice file to the terminal and plays it back to the user, who can use it to practice their presentation.

[0579] Step 12:

[0580] Users can use the virtual reality system to practice presentations in a specified scene. For example, they can wear an Oculus Quest 2 and simulate a virtual conference room or factory.

[0581] Step 13:

[0582] The server collects practice data in the virtual reality environment in real time and performs emotional analysis. Based on the user's movements, facial expressions, and tone of voice, real-time feedback is generated and presented to the user in the virtual reality environment.

[0583] Step 14:

[0584] The robot provides work instruction within the factory. Based on the generated presentation materials and feedback, the robot explains specific operating procedures and provides real-time instruction. By receiving instruction from the robot, users can improve their practical skills.

[0585] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0586] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0587] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0588] [Second embodiment]

[0589] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0590] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0591] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0592] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0593] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0594] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0595] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0596] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0597] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0598] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0599] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0600] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0601] The system of the present invention is designed to assist users who are not good at or have little experience creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[0602] Document creation function

[0603] User Action:

[0604] The user enters the content of the presentation into the designated input form. For example, if the user wants to present a plan to launch a new product into the market, the user writes the details of the presentation in the input form. Once the input is complete, the user clicks the "Submit" button.

[0605] The device:

[0606] The input presentation content is sent to the server, where it is properly formatted.

[0607] The server:

[0608] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. For example, it generates a title slide, product features slide, market analysis slide, introduction schedule slide, and conclusion slide based on the input topic, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the device.

[0609] The device:

[0610] The received presentation materials are displayed to the user, who can then review them and make corrections or additions as necessary.

[0611] Evaluation function

[0612] User Action:

[0613] Users record videos of their practice presentations and upload them to the system.

[0614] The device:

[0615] The recorded video file is sent to the server.

[0616] The server:

[0617] The server that receives the video performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, specific evaluations are made such as "speaking too fast" or "unfocused gaze," and feedback is generated based on the results. This feedback is then sent to the device.

[0618] The device:

[0619] The received feedback is displayed to the user, allowing them to see areas for improvement and correction, and to adjust their next practice based on this.

[0620] Demonstration function

[0621] User Action:

[0622] Users upload the presentation materials they have created and a sample of their own voice (audio file) to the system.

[0623] The device:

[0624] Send the uploaded data to the server.

[0625] The server:

[0626] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice samples. The server then uses the voice to demonstrate the presentation materials and generate a synthetic voice presentation. The generated audio file is then sent to the device.

[0627] The device:

[0628] The demonstration audio is played for users, who can use it as a model to improve their presentation content and speaking style.

[0629] Virtual practice function for each scene

[0630] User Action:

[0631] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[0632] The device:

[0633] Send scene setting information to the server.

[0634] The server:

[0635] The server receives the scene information and generates the specified virtual environment, and the virtual environment data is sent to the device and the VR goggles.

[0636] The device:

[0637] The received VR environment data is transferred to the VR goggles.

[0638] User Action:

[0639] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual large hall, they can gain experience in a situation similar to a real presentation.

[0640] As described above, the system of the present invention provides consistent support from the rapid creation of presentation materials to effective practice methods, thereby improving the presentation skills of users.

[0641] The processing flow will be explained below.

[0642] Document creation function

[0643] Processing Steps:

[0644] Step 1:

[0645] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[0646] Step 2:

[0647] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[0648] Step 3:

[0649] The device formats the presentation content it receives and sends it to the server.

[0650] Step 4:

[0651] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[0652] Step 5:

[0653] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[0654] Step 6:

[0655] Save the generated presentation materials in PDF or PPT format and send them to your device.

[0656] Step 7:

[0657] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[0658] Evaluation function

[0659] Processing Steps:

[0660] Step 1:

[0661] A user records a video of their practice presentation, which includes the user giving the presentation.

[0662] Step 2:

[0663] Users upload recorded videos to the system.

[0664] Step 3:

[0665] The device sends the uploaded video file to the server.

[0666] Step 4:

[0667] The server analyzes the received video, specifically, analyzing the audio and images.

[0668] Step 5:

[0669] The server extracts evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the speaker's speaking speed or their gaze is not focused.

[0670] Step 6:

[0671] The server generates feedback based on the analysis results, including suggestions for improvement and correction.

[0672] Step 7:

[0673] The server generates feedback and sends it to the device.

[0674] Step 8:

[0675] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[0676] Demonstration function

[0677] Processing Steps:

[0678] Step 1:

[0679] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[0680] Step 2:

[0681] The device sends the uploaded presentation materials and voice sample to the server.

[0682] Step 3:

[0683] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[0684] Step 4:

[0685] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[0686] Step 5:

[0687] The server transmits the generated demonstration audio file to the terminal.

[0688] Step 6:

[0689] The device displays a demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[0690] Virtual practice function for each scene

[0691] Processing Steps:

[0692] Step 1:

[0693] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[0694] Step 2:

[0695] The device sends the scene setting information to the server.

[0696] Step 3:

[0697] The server generates a specified virtual environment based on the scene information received.

[0698] Step 4:

[0699] The VR environment data generated by the server is sent to the terminal and VR goggles.

[0700] Step 5:

[0701] The terminal transfers the received VR environment data to the VR goggles.

[0702] Step 6:

[0703] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual hall, they can gain experience in a situation that is close to a real presentation.

[0704] Example 1

[0705] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Creating and delivering presentations is a difficult, time-consuming and laborious task for many users, placing a significant burden on beginners and those with little experience. This issue extends beyond support for creating presentation materials to the improvement of actual presentation skills. Conventional methods require time to create presentation materials and lack efficient practice methods, making it difficult to effectively improve presentation skills in a short period of time.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0708] In this invention, the server includes means for analyzing the presentation content and automatically generating presentation materials using a generative AI model, means for saving the generated presentation materials in PDF or PPT format and sending them to a terminal, and means for analyzing a presentation practice video and providing feedback based on the results. This allows for the rapid generation of presentation materials based on the content entered by the user, and further analysis of the practice video and provision of feedback, enabling effective improvement of presentation skills in a short period of time.

[0709] "User" refers to a person who uses the system to create or practice presentation materials.

[0710] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[0711] "Server" refers to the computer system that processes data, runs the generative AI model, stores and transmits generated materials, and provides analysis and feedback.

[0712] A "generative AI model" refers to an artificial intelligence model that automatically generates presentation materials based on input text data.

[0713] "Presentation materials" refers to slides, documents, etc. that visually display the contents of the presentation.

[0714] "Input form" refers to the interface that allows users to input the content of the presentation.

[0715] "Feedback" refers to evaluations and suggestions for improvement provided to users that are generated based on the results of analyzing the presentation practice video.

[0716] "Synthetic voice" refers to artificial voice data generated to resemble the user's voice.

[0717] "Virtual reality" refers to a simulated environment generated using computer technology.

[0718] "Simulation" refers to the near-realistic reproduction of a particular scenario or environment.

[0719] "Prompts" refer to guidelines or instructions used by generative AI models to generate presentation materials.

[0720] The system of the present invention is designed to assist users who are not good at creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[0721] First, the user enters the content of the presentation into a specified input form. For example, if a user wants to present a new product's market launch plan, they enter the details and click the "Submit" button. The device then converts the input presentation content into an appropriate format and sends it to the server using a protocol such as an HTTP POST request.

[0722] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (e.g., OpenAI's GPT-4). Specifically, based on the input theme, it generates a title slide, product features, market analysis, introduction schedule, and conclusion slides, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the terminal using file transfer protocols (FTP or HTTP).

[0723] The device displays the received presentation materials to the user, who can then review them and make corrections or additions as necessary.

[0724] Example prompt sentence:

[0725] "Please create a presentation about your new product launch plan. Key points should include product features, market analysis, launch schedule, and expected results."

[0726] Next, the user can record a video of the practice presentation and upload it to the system. The device sends the recorded video file to the server using a protocol such as an HTTP POST request.

[0727] After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the user's speaking speed or their eye gaze is not focused, and generate detailed feedback based on the results. The generated feedback is sent to the device in a text message format (e.g., JSON).

[0728] The device displays the received feedback to the user, allowing the user to see areas for improvement or correction.

[0729] In addition, users can upload their created presentation materials and a sample of their own voice (audio file) to the system. The device sends the data to the server, which uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice sample. The presentation materials are then demonstrated using this synthetic voice, and a synthetic voice presentation is generated. The generated presentation is then sent to the device.

[0730] The device plays back the received synthesized speech presentation, allowing the user to use it as a model to improve their presentation content and speaking style.

[0731] Finally, the user can select the VR practice mode and set the desired scene (e.g., a large hall or a conference room). The device sends the scene setting information to the server, which then generates the specified virtual environment. This virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine) and sent to the device and VR goggles.

[0732] The device then transfers the received VR environment data to the VR goggles, where the user can practice their presentation in the virtual environment. For example, by practicing a presentation in a virtual large hall, users can gain experience in a situation similar to a real presentation.

[0733] As described above, the system of the present invention consistently supports everything from the rapid creation of presentation materials to effective practice methods, thereby improving the user's presentation skills.

[0734] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0735] Step 1:

[0736] The user enters the details of the presentation in the designated input form. When the input is complete, the user clicks the "Send" button. The input is sent to the terminal as text data.

[0737] Step 2:

[0738] The terminal receives the text data entered by the user and formats it into an appropriate format (such as JSON). The formatted data is then sent to the server using an HTTP POST request. Specifically, the text data entered in the input form is converted into JSON format.

[0739] Step 3:

[0740] The server receives the JSON-formatted text data sent via an HTTP POST request, parses the received data, and extracts the theme and content of the presentation. During this analysis process, the analysis engine identifies keywords and context in the input data.

[0741] Step 4:

[0742] The server inputs the analyzed data into a generative AI model (e.g., GPT-4). The generative AI model automatically generates presentation materials based on the input data. The generative AI model generates slides containing content such as a title slide, product features, market analysis, introduction schedule, and conclusion.

[0743] Step 5:

[0744] The server saves the generated presentation slides in PDF or PPT format, and the saved files are sent to the terminal via HTTP or FTP. Specifically, the generated slides are converted to PDF format and saved in the specified directory.

[0745] Step 6:

[0746] The terminal receives the PDF or PPT file from the server and displays it to the user using a PDF reader or PPT viewer. The user can then review the document and make corrections or additions as necessary.

[0747] Step 7:

[0748] A user records a video of their practice presentation and uploads it to the system. The recorded video file is recognized by the device and sent to the server. Specifically, a video file is selected and uploaded to the server via an HTTP POST request.

[0749] Step 8:

[0750] The server receives the uploaded video file and performs audio and image analysis, extracting evaluation points such as speaking style, tone of voice, speed, eye movements, and posture.

[0751] Step 9:

[0752] The server generates feedback based on the analyzed evaluation points. The feedback is generated in text message format (e.g., JSON) and includes specific improvements and corrections. This feedback is sent to the device as an HTTP response.

[0753] Step 10:

[0754] The device analyzes the feedback received from the server and displays it to the user, who can then review the feedback and make improvements to their next practice session.

[0755] Step 11:

[0756] The user uploads the presentation materials they have created and a sample of their own voice (an audio file) to the system. The device sends the data to the server. Specifically, the presentation materials and audio file are selected and uploaded to the server via an HTTP POST request.

[0757] Step 12:

[0758] The server analyzes the received presentation materials and voice samples and uses a generative AI model to generate a synthetic voice that matches the content of the presentation materials.

[0759] Step 13:

[0760] The server then integrates the synthesized speech into the presentation materials to generate a live audio presentation, which is then saved as an audio file and sent to the device via HTTP or FTP.

[0761] Step 14:

[0762] The terminal plays the presentation with audio received from the server and displays it to the user, who can use it as a model to improve their own presentation skills.

[0763] Step 15:

[0764] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The device sends the setting information to the server. Specifically, the VR scene selection information is sent to the server in JSON format.

[0765] Step 16:

[0766] The server generates a virtual environment based on the VR scene information. The virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine).

[0767] Step 17:

[0768] The virtual environment data is sent from the server to the device and the VR goggles. The device then transfers the received VR environment data to the VR goggles. The user can then wear the VR goggles and practice their presentation in the virtual environment.

[0769] The above is the flow of program processing, including the specific operations performed at each step and an explanation of the inputs and outputs.

[0770] (Application example 1)

[0771] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0772] Creating presentation materials and practicing presentations is difficult and time-consuming for many users. Furthermore, in brick-and-mortar stores, staff are required to be able to give efficient presentations, such as when selling new products or introducing services, and the associated skills training is also important. Therefore, there is a need for a system that allows users to efficiently create presentation materials, practice, evaluate, and demonstrate them, and also improve their presentation skills in brick-and-mortar stores.

[0773] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0774] In this invention, the server includes a means for a user to input the content of a presentation, a means for automatically generating presentation materials based on the input content, a means for displaying the generated presentation materials, a means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, a means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and a means for generating a virtual environment within a physical store and practicing the presentation within that environment. This enables users to quickly create, evaluate, and receive feedback on their presentation materials, as well as provide effective training within the physical store.

[0775] "User" refers to an individual or organizational member who creates, evaluates, and practices presentation materials.

[0776] "Input means" refers to an interface or device that allows a user to provide specific content to a system.

[0777] "Automatic generation means" refers to software and algorithms for generating presentation materials based on specified inputs.

[0778] "Displaying means" refers to a display device or software that allows users to see the generated presentation materials and feedback.

[0779] "Video analysis means" means software and hardware for analyzing and evaluating the video and audio of a recorded presentation.

[0780] "Means for providing feedback" refers to interfaces and algorithms for providing users with suggestions for improvement and advice based on the analysis results.

[0781] "Means for demonstrating using synthetic speech" refers to technology that generates similar synthetic speech based on a user's voice sample and uses the speech sample to demonstrate the speech using presentation materials.

[0782] "Virtual reality methods" refers to the use of VR devices and software to create a virtual environment in which to practice a presentation.

[0783] "System" refers to the entire software and hardware collection that combines the above means to consistently support the creation, practice, and evaluation of presentation materials.

[0784] "Synthetic voice" refers to an artificial voice that is generated to resemble the user's voice.

[0785] "Virtual environment" refers to a computer-generated simulation that allows users to experience situations similar to those in the real world.

[0786] An "in-store" virtual environment refers to a virtual reality scene that mimics an actual in-store situation.

[0787] "Voice analysis" refers to the technology of analyzing recorded audio data to extract evaluation points such as tone of voice and speaking speed.

[0788] "Image analysis" refers to the technology of analyzing recorded video data to extract evaluation points such as eye movements and posture.

[0789] The system based on this invention basically comprises the following means: means for a user to input the content of the presentation, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and means for generating a virtual environment within a physical store and practicing the presentation in that environment.

[0790] User operations

[0791] First, the user logs in to the terminal and enters the content of the presentation into the designated input form. For example, if they are describing a market launch plan for a new product, they enter the details. Then, by clicking the "Submit" button, the input content is sent to the server.

[0792] Creating documents using generative AI models

[0793] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. For example, based on input about a new product's market launch plan, it generates a title slide, product features, market analysis, launch schedule, and conclusion slides. The materials are saved in PDF or PPT format and sent to the device.

[0794] View and modify materials

[0795] The terminal displays the generated material to the user, who can check it and make corrections or additions as necessary.

[0796] Presentation skills evaluation

[0797] Next, the user practices the presentation, records the video, and uploads it to the system. The device then sends the recorded video file to the server. After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. Specifically, evaluations are made on points such as "speaking too fast" and "unfocused eye contact," and feedback is generated based on the results. This feedback is sent to the device and displayed to the user. The user can use this information to improve their presentation skills.

[0798] Synthetic speech demonstration

[0799] Additionally, users upload their presentation materials and a sample of their own voice to the system. The server uses a generative AI model to generate a synthetic voice that resembles the user's voice. This voice is then used to create a demonstration voice using the presentation materials and sent to the device. This allows users to use the synthesized voice as a model to improve their presentation content and speaking style.

[0800] Virtual practice

[0801] The user selects the VR practice mode and sets the desired scene. For example, they can select a virtual environment such as a large hall, a conference room, or a brick-and-mortar store. The scene setting information is sent to the server, which then generates the specified virtual environment. The virtual environment data is then sent to the device and VR goggles. The user then puts on the VR goggles and practices their presentation in the virtual environment. This allows the user to gain experience in a situation that is close to an actual presentation scene.

[0802] Usage example

[0803] For example, if a store staff member were to prepare a presentation document that briefly explains the features of a new product, an "eco vacuum cleaner," they could use the system based on this invention. By inputting the following prompt sentence into the generative AI model, the document would be quickly created.

[0804] Example prompt sentence:

[0805] Create a sales presentation for an eco-friendly vacuum cleaner.

[0806] 1. Title Slide

[0807] 2. Product Features

[0808] 3. Market analysis

[0809] 4. Implementation Schedule

[0810] 5. Conclusion

[0811] In this way, by utilizing the system of the present invention, it is possible to quickly create presentation materials, practice and evaluate them effectively, and improve presentation skills in a physical store.

[0812] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0813] Step 1:

[0814] The user logs in to the terminal and enters the content of the presentation into the specified input form. The input form contains fields for describing details such as the presentation theme, purpose, and target audience, and the input is sent to the server by pressing the "Send" button. The input is in text format.

[0815] Step 2:

[0816] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. Specific prompts are passed to the generative AI model for analysis. For example, based on a new product's market launch plan, a title slide, product features, market analysis, launch schedule, and conclusions are generated. The generated slides are saved in PDF or PPT format and sent to the device.

[0817] Step 3:

[0818] The device displays the generated presentation materials to the user. The user can review the content based on these materials and make corrections or additions as necessary. The corrected data is also sent to the server, where it can be processed again by the generative AI model.

[0819] Step 4:

[0820] A user practices a presentation and records a video of it. Once the recording is complete, the user uploads the video file to the system. The device then sends the recorded video file to the server. The video file is in a common video format (e.g., MP4).

[0821] Step 5:

[0822] The server analyzes the received video file using audio and image analysis. Audio analysis evaluates voice tone and speaking speed, while image analysis evaluates eye movements and posture. These evaluation points are extracted and specific feedback (e.g., "You speak too fast" or "Your gaze is not focused") is generated. Based on this feedback, areas for improvement and correction are extracted and sent to the device.

[0823] Step 6:

[0824] The device displays feedback to the user, who can then take action to improve their presentation skills.

[0825] Step 7:

[0826] Users upload their created presentation materials and a sample of their own voice to the system. The server then uses a generative AI model to generate synthetic speech based on the user's voice sample. This synthetic speech is then used to demonstrate the presentation materials, providing the user with a model presentation to use.

[0827] Step 8:

[0828] The device plays a demonstration of the generated synthesized voice to the user, who can use this as a reference to further improve the content and speaking style of their presentation.

[0829] Step 9:

[0830] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, or a brick-and-mortar store). This scene information is sent to the server, which then generates the specified virtual environment.

[0831] Step 10:

[0832] The generated virtual environment data is sent to the VR goggles via the terminal. The user can then put on the VR goggles and practice their presentation in the virtual environment, allowing them to gain experience in a situation that is close to a real presentation.

[0833] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0834] The system of the present invention includes a means for a user to input the content of a presentation and automatically generate presentation materials using a generative AI model, a means for displaying the generated presentation materials, a means for analyzing a video of a practice presentation and providing feedback, a means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, a means for practicing the presentation in a scene set up using virtual reality, and a means for analyzing emotions by combining an emotion engine.

[0835] Document creation function

[0836] User Action:

[0837] The user enters the content of the presentation into a dedicated input form. For example, they can enter content such as "Marketing strategy for the next product." Once the input is complete, they click the "Submit" button.

[0838] The device:

[0839] The presentation content sent by the user is sent to the server. The input content is properly formatted.

[0840] The server:

[0841] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials, such as a title slide, marketing strategy details, market analysis, action plan, and conclusion slides.

[0842] The device:

[0843] The generated presentation materials are received in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[0844] Evaluation function

[0845] User Action:

[0846] Users record videos of their practice presentations and upload them to the system.

[0847] The device:

[0848] The recorded video file is sent to the server.

[0849] The server:

[0850] The received video is analyzed. Specifically, audio analysis, image analysis, and an emotion engine are used to analyze the user's emotions.

[0851] The server:

[0852] Based on the analysis results, the system evaluates the presentation's delivery style, tone of voice, speed, eye movements, posture, and even emotional aspects, and generates feedback, including specific suggestions for improvement such as "adjust your speaking speed" and "maintain a consistent gaze."

[0853] The device:

[0854] Feedback is received and displayed to the user, who can use it to improve their next practice.

[0855] Demonstration function

[0856] User Action:

[0857] Users upload their presentation materials and a sample of their voice.

[0858] The device:

[0859] These data are sent to the server.

[0860] The server:

[0861] Based on the presentation materials and audio samples, a generative AI model is used to generate a synthetic voice that resembles the user's voice. This synthetic voice is then used to generate a demonstration audio file using the presentation materials.

[0862] The device:

[0863] The system receives a demonstration audio file and plays it for the user, who can use it as a model to improve their presentation content and speaking style.

[0864] Virtual practice function for each scene

[0865] User Action:

[0866] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[0867] The device:

[0868] Send scene setting information to the server.

[0869] The server:

[0870] The specified virtual environment is generated based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[0871] The device:

[0872] The received VR environment data is transferred to the VR goggles.

[0873] User Action:

[0874] Users put on VR goggles and practice their presentations in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[0875] The role of the emotional engine

[0876] The server:

[0877] The server uses an emotion engine to collect emotional data from the user's facial expressions, tone of voice, and speaking style during presentation practice videos and real-time VR practice sessions, detecting emotions such as tension, joy, and impatience.

[0878] The server:

[0879] The emotion engine analyzes the detected emotional data and generates feedback based on it, including specific advice such as "You seem nervous, so try slowing down your speech" or "Speak more emotively."

[0880] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[0881] The processing flow will be explained below.

[0882] Document creation function

[0883] Processing Steps:

[0884] Step 1:

[0885] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[0886] Step 2:

[0887] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[0888] Step 3:

[0889] The device formats the presentation content it receives and sends it to the server.

[0890] Step 4:

[0891] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[0892] Step 5:

[0893] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[0894] Step 6:

[0895] Save the generated presentation materials in PDF or PPT format and send them to your device.

[0896] Step 7:

[0897] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[0898] Evaluation function

[0899] Processing Steps:

[0900] Step 1:

[0901] A user records a video of their practice presentation, which includes the user giving the presentation.

[0902] Step 2:

[0903] Users upload recorded videos to the system.

[0904] Step 3:

[0905] The device sends the uploaded video file to the server.

[0906] Step 4:

[0907] The server analyzes the received video. Specifically, it performs audio and image analysis, and then extracts emotional data using an emotion engine.

[0908] Step 5:

[0909] The server evaluates speaking style, tone of voice, speed, eye movements, posture, and emotions, and extracts evaluation points, such as "speaking too fast," "unfocused eye contact," and "appears nervous."

[0910] Step 6:

[0911] The server generates feedback based on the analysis results. The feedback includes suggestions for improving speaking style and posture, as well as emotional advice. For example, "You seem nervous, so try taking a deep breath" or "Try to lower your voice tone and speak calmly."

[0912] Step 7:

[0913] The server generates feedback and sends it to the device.

[0914] Step 8:

[0915] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[0916] Demonstration function

[0917] Processing Steps:

[0918] Step 1:

[0919] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[0920] Step 2:

[0921] The device sends the uploaded presentation materials and voice sample to the server.

[0922] Step 3:

[0923] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[0924] Step 4:

[0925] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[0926] Step 5:

[0927] The server transmits the generated demonstration audio file to the terminal.

[0928] Step 6:

[0929] The device plays the demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[0930] Virtual practice function for each scene

[0931] Processing Steps:

[0932] Step 1:

[0933] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[0934] Step 2:

[0935] The device sends the scene setting information to the server.

[0936] Step 3:

[0937] The server generates a specified virtual environment based on the scene information received.

[0938] Step 4:

[0939] The VR environment data generated by the server is sent to the terminal and VR goggles.

[0940] Step 5:

[0941] The terminal transfers the received VR environment data to the VR goggles.

[0942] Step 6:

[0943] The user puts on VR goggles and practices their presentation in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[0944] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[0945] Example 2

[0946] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0947] Traditional methods for creating, practicing, and evaluating presentation materials require a great deal of time and effort, making it difficult to improve the quality of presentations. In particular, in order to receive feedback on one's own performance, one must request evaluations from others, making it difficult to receive objective and continuous feedback. Furthermore, the content and structure of presentation materials must be created manually, making it difficult to generate materials quickly and effectively.

[0948] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0949] In this invention, the server includes: means for a user to input presentation content; means for using a generative AI model to automatically generate presentation materials based on the input content; means for displaying the generated presentation materials; means for analyzing a video of a practice presentation and providing feedback based on the results using audio analysis, image analysis, and the generative AI model; means for demonstrating a model presentation using a synthesized voice that resembles the user's voice; means for practicing the presentation in a scene set using virtual reality; and means for analyzing emotional data during the presentation using an emotion engine and providing feedback. This allows users to quickly and automatically create presentation materials and receive multifaceted feedback to improve the quality of their presentations.

[0950] "Presentation content" refers to the information and data that a user inputs to create presentation materials.

[0951] A "generative AI model" is an artificial intelligence algorithm that automatically creates presentation materials based on input content.

[0952] "Presentation Materials" means the content of a presentation in slide or other format, typically generated as a PDF or PPT file.

[0953] "Voice analysis" is a technology that analyzes recorded voice data and evaluates the tone, speed, quality of the voice, etc.

[0954] "Image analysis" is a technology that analyzes recorded video data and evaluates eye movements, facial expressions, posture, etc.

[0955] "Feedback" is an evaluation of the content and performance of a presentation and specific advice for improvement.

[0956] "Synthetic voice" is an artificial voice that resembles the user's voice and is generated based on a user's voice sample.

[0957] "Virtual reality" refers to a virtual digital environment generated using computer technology in which users can interact.

[0958] The "emotion engine" is a technology that analyzes a user's emotions from facial expressions, tone of voice, etc., and detects specific emotional states.

[0959] A "scene" refers to a specific environment or location within virtual reality where a presentation takes place.

[0960] MODE FOR CARRYING OUT THE INVENTION

[0961] The system of the present invention automatically generates presentation materials based on the content entered by the user in order to improve the quality of presentations, and provides multifunctional support for supporting presentation practice.

[0962] Document creation function

[0963] User operations

[0964] The user enters the content of the presentation into a dedicated input form, for example, a specific content such as "Marketing strategy for the next product," and clicks the submit button.

[0965] Device operation

[0966] The device formats the input presentation content appropriately and sends it to the server. The data sent is in text format and is encoded.

[0967] Server Operations

[0968] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. Specifically, slides are generated, including a title slide, details of the marketing strategy, market analysis, an action plan, and a conclusion. Various natural language processing technologies are used in this generative AI model.

[0969] Device operation

[0970] The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[0971] Prompt Sentence Examples

[0972] "Please create a presentation about the marketing strategy for our next product."

[0973] Evaluation function

[0974] User operations

[0975] The user records a video of the presentation practice and uploads the video file to the system.

[0976] Device operation

[0977] The device sends the recorded video file to the server, where the file format and size are checked.

[0978] Server Operations

[0979] The server analyzes the received video using audio analysis, image analysis, and an emotion engine, and analyzes voice tone, speed, eye movement, posture, emotion, etc.

[0980] Server Operations

[0981] Feedback is generated based on the analysis results, including suggestions for improvement such as "adjust speaking speed" and "maintain consistent eye contact."

[0982] Device operation

[0983] The feedback is sent to the device and displayed to the user, who can use it to improve their next practice.

[0984] Prompt Sentence Examples

[0985] "Record yourself practicing your presentation and provide feedback."

[0986] Demonstration function

[0987] User operations

[0988] Users upload their own presentation materials and voice samples to the system.

[0989] Device operation

[0990] Your presentation and audio samples are sent to the server, where they are checked for correct file format and size.

[0991] Server Operations

[0992] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the uploaded voice sample, and then uses the generated synthetic voice to create a demonstration audio file of the presentation material.

[0993] Device operation

[0994] The demonstration audio file is sent to the terminal and played back for the user, who can use the audio file as a model to improve the content and speaking style of their presentation.

[0995] Prompt Sentence Examples

[0996] "Please convert your presentation materials into audio files and play them as a model."

[0997] Virtual practice function for each scene

[0998] User operations

[0999] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). Scenes can be selected from the system's catalog.

[1000] Device operation

[1001] Scene setting information is sent to the server, including the selected scene's ID and detailed parameters.

[1002] Server Operations

[1003] The server generates the specified virtual environment based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[1004] Device operation

[1005] The terminal transfers the received VR environment data to the VR goggles.

[1006] User operations

[1007] Users put on VR goggles and practice their presentation in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data.

[1008] Prompt Sentence Examples

[1009] "I want to practice my presentation in a virtual conference room environment."

[1010] By combining these functions, users can quickly create presentation materials and practice them with detailed feedback. Furthermore, by using an emotion engine, it is possible to analyze emotions such as tension and joy and provide more effective feedback.

[1011] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1012] Document creation function

[1013] Step 1:

[1014] User actions

[1015] The user enters the content of the presentation into a dedicated input form and clicks the send button. The input is specific content such as "marketing strategy for the next product." The output is the content of the form being sent to the terminal as data.

[1016] Step 2:

[1017] Processing performed by the device

[1018] The device properly formats the presentation content received from the user and sends it to the server. The input is the text data of the presentation content sent by the user, and the output is sending the formatted data to the server. Specifically, the data is formatted, encoded, and sent to the server according to the communication protocol.

[1019] Step 3:

[1020] Processing performed by the server

[1021] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. The input is the formatted presentation content, and the output is the generated presentation material. Specific operations include text analysis, determining the slide structure and layout, and generating automatically inserted images and graphs.

[1022] Step 4:

[1023] Processing performed by the device

[1024] The terminal receives the generated presentation materials in PDF or PPT format and displays them to the user. The input is the presentation material file sent from the server, and the output is to display that file on the user's device. Specific operations include downloading the file, displaying it in a viewer, and accepting edits.

[1025] Evaluation function

[1026] Step 1:

[1027] User actions

[1028] A user records a video of a presentation practice and uploads the video file to the system. The input is the recorded video file, and the output is the uploaded video data.

[1029] Step 2:

[1030] Processing performed by the device

[1031] The device sends the recorded video file to the server. The input is the video file uploaded by the user, and the output is to send this file to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[1032] Step 3:

[1033] Processing performed by the server

[1034] The server uses audio analysis, image analysis, and an emotion engine to analyze the received video. The input is the received video file, and the output is the analysis results and feedback. Specific operations include analyzing voice tone and speed, analyzing eye movements and posture, and detecting emotional states.

[1035] Step 4:

[1036] Processing performed by the server

[1037] Feedback is generated based on the analysis results. The input is the analysis results, and the output is the generated feedback. Specific operations include extracting areas for improvement based on the analysis data and generating specific feedback statements.

[1038] Step 5:

[1039] Processing performed by the device

[1040] The terminal receives the feedback data and displays it to the user. The input is the feedback data sent from the server, and the output is to show this data to the user. Specific operations include displaying feedback text, hints, and operation guides for the user's next practice.

[1041] Demonstration function

[1042] Step 1:

[1043] User actions

[1044] Users upload their created presentation materials and their own voice samples to the system. The input is the presentation material file and the voice sample file, and the output is the transfer of these data to the terminal.

[1045] Step 2:

[1046] Processing performed by the device

[1047] The terminal sends presentation materials and audio samples to the server. The input is the file uploaded by the user, and the output is to send these files to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[1048] Step 3:

[1049] Processing performed by the server

[1050] The server uses a generative AI model to generate synthetic speech that resembles the user's voice based on the uploaded voice sample. This synthetic speech is then used to create a demonstration audio file of the presentation materials. The input is the voice sample and the presentation materials, and the output is the synthesized demonstration audio file. Specific operations include speech synthesis, audio file generation, and reading of the presentation materials.

[1051] Step 4:

[1052] Processing performed by the device

[1053] Receives a demonstration audio file and plays it for the user. The input is the audio file sent from the server, and the output is the playback of the audio file. Specific operations include downloading the file, playing it in an audio player, and accepting user operations.

[1054] Virtual practice function for each scene

[1055] Step 1:

[1056] User actions

[1057] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The input is the scene setting information, and the output is the set information being sent to the device.

[1058] Step 2:

[1059] Processing performed by the device

[1060] Sends scene setting information to the server. The input is the scene information set by the user, and the output is to send this information to the server. Specific operations include formatting the setting information and communicating it according to the transmission protocol.

[1061] Step 3:

[1062] Processing performed by the server

[1063] The server generates a virtual environment based on the received scene information. The input is the received scene information, and the output is the generated virtual environment data. Specific operations include 3D modeling of the virtual environment and scenario setting.

[1064] Step 4:

[1065] Processing performed by the device

[1066] The received VR environment data is transmitted to the VR goggles. The input is the virtual environment data sent from the server, and the output is the data transmitted to the VR goggles. The specific operations include decoding the data and transmitting it to the device.

[1067] Step 5:

[1068] User actions

[1069] Users put on VR goggles and practice their presentations in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data. The input is the actual presentation's movements and emotional data, and the output is real-time feedback.

[1070] The above are the processing steps in the system of the present invention, which allows users to achieve comprehensive improvement in their presentation skills.

[1071] (Application example 2)

[1072] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1073] Conventional presentation practice support systems incorporate functions for automatically generating presentation materials and providing feedback on practice, but they are not adequately suited to real-world work environments, particularly in factory environments. Furthermore, because they do not provide real-time feedback based on emotional data, it is difficult to effectively alleviate issues such as tension and impatience. This results in problems such as reduced user work efficiency and reduced teaching effectiveness.

[1074] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1075] In this invention, the server includes means for a user to input presentation content, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for analyzing a video of a practice presentation and providing feedback based on the results, means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, means for practicing the presentation in a scene set using virtual reality, means for analyzing a recorded video of the practice presentation, collecting emotional data and providing feedback, and means for providing work guidance in a factory using virtual reality and robots. This allows users to practice in a simulation that resembles an actual work environment, and by receiving feedback based on the emotional data, the quality of their presentations can be improved.

[1076] "User" refers to a person who uses the presentation system to create presentation materials, practice them, and receive feedback.

[1077] "Presentation content" refers to the information and outline that a user enters when creating presentation materials.

[1078] "Auto-generation" refers to the process of automatically generating presentation materials using machine learning models and algorithms based on input information and prompts.

[1079] "Presentation materials" refers to materials such as slides and documents used to effectively communicate the content of a presentation.

[1080] "Display means" refers to a device or method for visually presenting the generated presentation materials to the user.

[1081] "Feedback" means information or comments that provide improvements or evaluations based on the results of your practice presentation.

[1082] "Synthetic voice" refers to a voice that resembles a human voice and is generated using voice synthesis technology.

[1083] "Virtual reality" means a virtual environment created using computer graphics and other technologies, in which users can have a realistic experience.

[1084] "Emotional data" refers to information about the user's emotional state analyzed from facial expressions, voice, etc.

[1085] "Work instruction" means the process of providing instruction and training regarding work tasks and procedures within a factory.

[1086] A "robot" refers to an automated machine that performs programmed actions and provides information and work instructions to users.

[1087] The system of the present invention allows users to input presentation content, automatically generate presentation materials using a generative AI model, display and evaluate the materials, and practice the presentation in a virtual reality environment. To effectively implement the system of the present invention, the following hardware and software are used:

[1088] Hardware and software used

[1089] 1. Hardware

[1090] Server: High-performance data processing server

[1091] Device: A computer or tablet that a user operates.

[1092] VR system: Virtual reality goggles such as Oculus Quest 2

[1093] Robot: A multi-functional robot that provides guidance and feedback within the factory

[1094] 2. Software

[1095] Generative AI model: OpenAI GPT-4

[1096] Sentiment analysis model: Custom trained model using TensorFlow and Keras

[1097] Video analysis software: OpenCV and MoviePy

[1098] Voice analysis software: Librosa

[1099] Data Transfer and Management System: Cloud Database

[1100] System processing overview

[1101] First, the user inputs the content of the presentation using a dedicated terminal. For example, they input a prompt such as "Operating procedure for new equipment." Once the input is complete, the terminal sends the content to the server.

[1102] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (OpenAI GPT-4). These materials include a title slide, operating procedures, safety precautions, etc. The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user.

[1103] Next, the user records a video of their practice presentation and uploads it to the server via their device. The server receives the video, performs audio analysis (Librosa) and image analysis (OpenCV), and extracts emotional data using emotion analysis models (TensorFlow and Keras). This allows the server to determine emotional states such as tension or joy, and generates feedback based on the results.

[1104] Furthermore, a synthesized voice generated from a sample of the user's voice is created to demonstrate the presentation materials and provided to the user, allowing the user to learn from the model presentation.

[1105] Additionally, using a virtual reality system (Oculus Quest 2), users can practice their presentations in a designated scene (e.g., a specific location in a factory). The server generates this virtual environment and provides real-time feedback based on emotional data.

[1106] Finally, the robot will provide work instructions within the factory, explaining specific operating procedures and providing real-time guidance based on presentation materials and feedback.

[1107] Specific examples

[1108] For example, when leading a presentation on new safety procedures in a factory, a worker inputs the prompt phrase "new safety procedure." The generative AI model automatically creates presentation materials and displays them on the device. The worker can then give a presentation based on these materials and automatically receive feedback from the recorded video.

[1109] Prompt Sentence Examples

[1110] Prompt: "Operating procedures for new equipment"

[1111] Generated presentation contents:

[1112] Title: New equipment operating procedures

[1113] Slide 1: Facility Overview

[1114] Slide 2: Operational Procedure 1

[1115] Slide 3: Operational Procedure 2

[1116] Slide 4: Safety Precautions

[1117] Slide 5: Questions and Answers

[1118] In this way, the system of the present invention can comprehensively improve users' presentation skills and work efficiency through the rapid creation of presentation materials, evaluation of practice, provision of advanced feedback using emotional data, and practical instruction using virtual reality and robots.

[1119] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1120] Step 1:

[1121] The user inputs the content of the presentation. The user enters a prompt sentence into a dedicated input form and clicks the submit button. For example, the user enters "Operating procedures for new equipment." This input becomes the initial data for the system.

[1122] Step 2:

[1123] The device sends the input presentation content to the server. The device then formats the input content and sends it to the server in a format that is easy for the generative AI model to analyze. This is where data is sent and received.

[1124] Step 3:

[1125] Based on the presentation content received by the server, a generative AI model (OpenAI GPT-4) is used to automatically generate presentation materials. The generative AI model analyzes the input prompts and creates slides (PDF or PPT format) to present relevant information, such as a title slide, operating procedures, and safety precautions.

[1126] Step 4:

[1127] The server sends the generated presentation materials to the terminal. The generated materials are transferred to the terminal in the appropriate format (PDF or PPT) and displayed to the user. The user can check the materials and make corrections as necessary.

[1128] Step 5:

[1129] The user records a video of their practice presentation and uploads it to the server via their device. The recorded video is sent to the server and becomes the data for the next analysis step.

[1130] Step 6:

[1131] The server analyzes the received video. Specifically, audio analysis (Librosa) and image analysis (OpenCV) are performed, and emotional data is extracted using an emotion analysis model (TensorFlow and Keras). For example, MFCC features are extracted to determine the emotional state. This allows emotions such as tension and joy to be detected.

[1132] Step 7:

[1133] The server generates feedback based on the analysis results, including adjustments to speaking speed and improvements to eye contact. The server creates the feedback information and notifies the device.

[1134] Step 8:

[1135] The device displays the received feedback to the user, who can review the feedback and receive guidance on how to improve the presentation's delivery and content.

[1136] Step 9:

[1137] The user uploads a sample of their voice to the server, which then serves as input for generating the synthesized voice.

[1138] Step 10:

[1139] The server generates synthetic speech. Using a generative AI model, it generates synthetic speech that resembles the user's voice. Based on this synthetic speech, a model presentation audio file is created.

[1140] Step 11:

[1141] The server sends the synthesized voice file to the terminal and plays it back to the user, who can use it to practice their presentation.

[1142] Step 12:

[1143] Users can use the virtual reality system to practice presentations in a specified scene. For example, they can wear an Oculus Quest 2 and simulate a virtual conference room or factory.

[1144] Step 13:

[1145] The server collects practice data in the virtual reality environment in real time and performs emotional analysis. Based on the user's movements, facial expressions, and tone of voice, real-time feedback is generated and presented to the user in the virtual reality environment.

[1146] Step 14:

[1147] The robot provides work instruction within the factory. Based on the generated presentation materials and feedback, the robot explains specific operating procedures and provides real-time instruction. By receiving instruction from the robot, users can improve their practical skills.

[1148] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1149] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1150] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1151] [Third embodiment]

[1152] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1153] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1154] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1155] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1156] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1157] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1158] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1159] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1160] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1161] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1162] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1163] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1164] The system of the present invention is designed to assist users who are not good at or have little experience creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[1165] Document creation function

[1166] User Action:

[1167] The user enters the content of the presentation into the designated input form. For example, if the user wants to present about the market launch plan of a new product, the user writes the content in detail in the input form. Once the input is complete, the user clicks the "Submit" button.

[1168] The device:

[1169] The input presentation content is sent to the server, where it is properly formatted.

[1170] The server:

[1171] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. For example, it generates a title slide, product features slide, market analysis slide, introduction schedule slide, and conclusion slide based on the input topic, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the device.

[1172] The device:

[1173] The received presentation materials are displayed to the user, who can then review them and make corrections or additions as necessary.

[1174] Evaluation function

[1175] User Action:

[1176] Users record videos of their practice presentations and upload them to the system.

[1177] The device:

[1178] The recorded video file is sent to the server.

[1179] The server:

[1180] The server that receives the video performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, specific evaluations are made such as "speaking too fast" or "unfocused gaze," and feedback is generated based on the results. This feedback is then sent to the device.

[1181] The device:

[1182] The received feedback is displayed to the user, allowing them to see areas for improvement and correction, and to adjust their next practice based on this.

[1183] Demonstration function

[1184] User Action:

[1185] Users upload the presentation materials they have created and a sample of their own voice (audio file) to the system.

[1186] The device:

[1187] Send the uploaded data to the server.

[1188] The server:

[1189] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice samples. The server then uses the voice to demonstrate the presentation materials and generate a synthetic voice presentation. The generated audio file is then sent to the device.

[1190] The device:

[1191] The demonstration audio is played for users, who can use it as a model to improve their presentation content and speaking style.

[1192] Virtual practice function for each scene

[1193] User Action:

[1194] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[1195] The device:

[1196] Send scene setting information to the server.

[1197] The server:

[1198] The server receives the scene information and generates the specified virtual environment, and the virtual environment data is sent to the device and the VR goggles.

[1199] The device:

[1200] The received VR environment data is transferred to the VR goggles.

[1201] User Action:

[1202] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual large hall, they can gain experience in a situation similar to a real presentation.

[1203] As described above, the system of the present invention provides consistent support from the rapid creation of presentation materials to effective practice methods, thereby improving the presentation skills of users.

[1204] The processing flow will be explained below.

[1205] Document creation function

[1206] Processing Steps:

[1207] Step 1:

[1208] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[1209] Step 2:

[1210] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[1211] Step 3:

[1212] The device formats the presentation content it receives and sends it to the server.

[1213] Step 4:

[1214] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[1215] Step 5:

[1216] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[1217] Step 6:

[1218] Save the generated presentation materials in PDF or PPT format and send them to your device.

[1219] Step 7:

[1220] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[1221] Evaluation function

[1222] Processing Steps:

[1223] Step 1:

[1224] A user records a video of their practice presentation, which includes the user giving the presentation.

[1225] Step 2:

[1226] Users upload recorded videos to the system.

[1227] Step 3:

[1228] The device sends the uploaded video file to the server.

[1229] Step 4:

[1230] The server analyzes the received video, specifically, analyzing the audio and images.

[1231] Step 5:

[1232] The server extracts evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the speaker's speaking speed or their gaze is not focused.

[1233] Step 6:

[1234] The server generates feedback based on the analysis results, including suggestions for improvement and correction.

[1235] Step 7:

[1236] The server generates feedback and sends it to the device.

[1237] Step 8:

[1238] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[1239] Demonstration function

[1240] Processing Steps:

[1241] Step 1:

[1242] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[1243] Step 2:

[1244] The device sends the uploaded presentation materials and voice sample to the server.

[1245] Step 3:

[1246] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[1247] Step 4:

[1248] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[1249] Step 5:

[1250] The server transmits the generated demonstration audio file to the terminal.

[1251] Step 6:

[1252] The device displays a demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[1253] Virtual practice function for each scene

[1254] Processing Steps:

[1255] Step 1:

[1256] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[1257] Step 2:

[1258] The device sends the scene setting information to the server.

[1259] Step 3:

[1260] The server generates a specified virtual environment based on the scene information received.

[1261] Step 4:

[1262] The VR environment data generated by the server is sent to the terminal and VR goggles.

[1263] Step 5:

[1264] The terminal transfers the received VR environment data to the VR goggles.

[1265] Step 6:

[1266] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual hall, they can gain experience in a situation that is close to a real presentation.

[1267] Example 1

[1268] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1269] Creating and delivering presentations is a difficult, time-consuming and laborious task for many users, placing a significant burden on beginners and those with little experience. This issue extends beyond support for creating presentation materials to the improvement of actual presentation skills. Conventional methods require time to create presentation materials and lack efficient practice methods, making it difficult to effectively improve presentation skills in a short period of time.

[1270] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1271] In this invention, the server includes means for analyzing the presentation content and automatically generating presentation materials using a generative AI model, means for saving the generated presentation materials in PDF or PPT format and sending them to a terminal, and means for analyzing a presentation practice video and providing feedback based on the results. This allows for the rapid generation of presentation materials based on the content entered by the user, and further analysis of the practice video and provision of feedback, enabling effective improvement of presentation skills in a short period of time.

[1272] "User" refers to a person who uses the system to create or practice presentation materials.

[1273] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[1274] "Server" refers to the computer system that processes data, runs the generative AI model, stores and transmits generated materials, and provides analysis and feedback.

[1275] A "generative AI model" refers to an artificial intelligence model that automatically generates presentation materials based on input text data.

[1276] "Presentation materials" refers to slides, documents, etc. that visually display the contents of the presentation.

[1277] "Input form" refers to the interface that allows users to input the content of the presentation.

[1278] "Feedback" refers to evaluations and suggestions for improvement provided to users that are generated based on the results of analyzing the presentation practice video.

[1279] "Synthetic voice" refers to artificial voice data generated to resemble the user's voice.

[1280] "Virtual reality" refers to a simulated environment generated using computer technology.

[1281] "Simulation" refers to the near-realistic reproduction of a particular scenario or environment.

[1282] "Prompts" refer to guidelines or instructions used by generative AI models to generate presentation materials.

[1283] The system of the present invention is designed to assist users who are not good at creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[1284] First, the user enters the content of the presentation into a specified input form. For example, if a user wants to present a new product's market launch plan, they enter the details and click the "Submit" button. The device then converts the input presentation content into an appropriate format and sends it to the server using a protocol such as an HTTP POST request.

[1285] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (e.g., OpenAI's GPT-4). Specifically, based on the input theme, it generates a title slide, product features, market analysis, introduction schedule, and conclusion slides, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the terminal using file transfer protocols (FTP or HTTP).

[1286] The device displays the received presentation materials to the user, who can then review them and make corrections or additions as necessary.

[1287] Example prompt sentence:

[1288] "Please create a presentation about your new product launch plan. Key points should include product features, market analysis, launch schedule, and expected results."

[1289] Next, the user can record a video of the practice presentation and upload it to the system. The device sends the recorded video file to the server using a protocol such as an HTTP POST request.

[1290] After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the user's speaking speed or their eye gaze is not focused, and generate detailed feedback based on the results. The generated feedback is sent to the device in a text message format (e.g., JSON).

[1291] The device displays the received feedback to the user, allowing the user to see areas for improvement or correction.

[1292] In addition, users can upload their created presentation materials and a sample of their own voice (audio file) to the system. The device sends the data to the server, which uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice sample. The presentation materials are then demonstrated using this synthetic voice, and a synthetic voice presentation is generated. The generated presentation is then sent to the device.

[1293] The device plays back the received synthesized speech presentation, allowing the user to use it as a model to improve their presentation content and speaking style.

[1294] Finally, the user can select the VR practice mode and set the desired scene (e.g., a large hall or a conference room). The device sends the scene setting information to the server, which then generates the specified virtual environment. This virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine) and sent to the device and VR goggles.

[1295] The device then transfers the received VR environment data to the VR goggles, where the user can practice their presentation in the virtual environment. For example, by practicing a presentation in a virtual large hall, users can gain experience in a situation similar to a real presentation.

[1296] As described above, the system of the present invention consistently supports everything from the rapid creation of presentation materials to effective practice methods, thereby improving the user's presentation skills.

[1297] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1298] Step 1:

[1299] The user enters the details of the presentation in the designated input form. When the input is complete, the user clicks the "Send" button. The input is sent to the terminal as text data.

[1300] Step 2:

[1301] The terminal receives the text data entered by the user and formats it into an appropriate format (such as JSON). The formatted data is then sent to the server using an HTTP POST request. Specifically, the text data entered in the input form is converted into JSON format.

[1302] Step 3:

[1303] The server receives the JSON-formatted text data sent via an HTTP POST request, parses the received data, and extracts the theme and content of the presentation. During this analysis process, the analysis engine identifies keywords and context in the input data.

[1304] Step 4:

[1305] The server inputs the analyzed data into a generative AI model (e.g., GPT-4). The generative AI model automatically generates presentation materials based on the input data. The generative AI model generates slides containing content such as a title slide, product features, market analysis, introduction schedule, and conclusion.

[1306] Step 5:

[1307] The server saves the generated presentation slides in PDF or PPT format, and the saved files are sent to the terminal via HTTP or FTP. Specifically, the generated slides are converted to PDF format and saved in the specified directory.

[1308] Step 6:

[1309] The terminal receives the PDF or PPT file from the server and displays it to the user using a PDF reader or PPT viewer. The user can then review the document and make corrections or additions as necessary.

[1310] Step 7:

[1311] A user records a video of their practice presentation and uploads it to the system. The recorded video file is recognized by the device and sent to the server. Specifically, a video file is selected and uploaded to the server via an HTTP POST request.

[1312] Step 8:

[1313] The server receives the uploaded video file and performs audio and image analysis, extracting evaluation points such as speaking style, tone of voice, speed, eye movements, and posture.

[1314] Step 9:

[1315] The server generates feedback based on the analyzed evaluation points. The feedback is generated in text message format (e.g., JSON) and includes specific improvements and corrections. This feedback is sent to the device as an HTTP response.

[1316] Step 10:

[1317] The device analyzes the feedback received from the server and displays it to the user, who can then review the feedback and be aware of areas for improvement in their next practice session.

[1318] Step 11:

[1319] The user uploads the presentation materials they have created and a sample of their own voice (an audio file) to the system. The device sends the data to the server. Specifically, the presentation materials and audio file are selected and uploaded to the server via an HTTP POST request.

[1320] Step 12:

[1321] The server analyzes the received presentation materials and voice samples and uses a generative AI model to generate a synthetic voice that matches the content of the presentation materials.

[1322] Step 13:

[1323] The server then integrates the synthesized speech into the presentation materials to generate a live audio presentation, which is then saved as an audio file and sent to the device via HTTP or FTP.

[1324] Step 14:

[1325] The terminal plays the presentation with audio received from the server and displays it to the user, who can use it as a model to improve their own presentation skills.

[1326] Step 15:

[1327] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The device sends the setting information to the server. Specifically, the VR scene selection information is sent to the server in JSON format.

[1328] Step 16:

[1329] The server generates a virtual environment based on the VR scene information. The virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine).

[1330] Step 17:

[1331] The virtual environment data is sent from the server to the device and the VR goggles. The device then transfers the received VR environment data to the VR goggles. The user can then wear the VR goggles and practice their presentation in the virtual environment.

[1332] The above is the flow of program processing, including the specific operations performed at each step and an explanation of the inputs and outputs.

[1333] (Application example 1)

[1334] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1335] Creating presentation materials and practicing presentations is difficult and time-consuming for many users. Furthermore, in brick-and-mortar stores, staff are required to be able to give efficient presentations, such as when selling new products or introducing services, and the associated skills training is also important. Therefore, there is a need for a system that allows users to efficiently create presentation materials, practice, evaluate, and demonstrate them, and also improve their presentation skills in brick-and-mortar stores.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1337] In this invention, the server includes a means for a user to input the content of a presentation, a means for automatically generating presentation materials based on the input content, a means for displaying the generated presentation materials, a means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, a means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and a means for generating a virtual environment within a physical store and practicing the presentation within that environment. This enables users to quickly create, evaluate, and receive feedback on their presentation materials, as well as provide effective training within the physical store.

[1338] "User" refers to an individual or organizational member who creates, evaluates, and practices presentation materials.

[1339] "Input means" refers to an interface or device that allows a user to provide specific content to a system.

[1340] "Automatic generation means" refers to software and algorithms for generating presentation materials based on specified inputs.

[1341] "Means for displaying" refers to a display device or software that allows users to see the generated presentation materials and feedback.

[1342] "Video analysis means" means software and hardware for analyzing and evaluating the video and audio of a recorded presentation.

[1343] "Means for providing feedback" refers to interfaces and algorithms for providing users with suggestions for improvement or advice based on the analysis results.

[1344] "Means for demonstrating using synthetic speech" refers to technology that generates similar synthetic speech based on a user's voice sample and uses the speech sample to demonstrate the speech using presentation materials.

[1345] "Virtual reality methods" refers to the use of VR devices and software to create a virtual environment in which to practice a presentation.

[1346] "System" refers to the entire software and hardware collection that combines the above means to consistently support the creation, practice, and evaluation of presentation materials.

[1347] "Synthetic voice" refers to an artificial voice that is generated to resemble the user's voice.

[1348] "Virtual environment" refers to a computer-generated simulation that allows users to experience situations similar to those in the real world.

[1349] An "in-store" virtual environment refers to a virtual reality scene that mimics an actual in-store situation.

[1350] "Voice analysis" refers to the technology of analyzing recorded audio data to extract evaluation points such as tone of voice and speaking speed.

[1351] "Image analysis" refers to the technology of analyzing recorded video data to extract evaluation points such as eye movements and posture.

[1352] The system based on this invention basically comprises the following means: means for a user to input the content of the presentation, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and means for generating a virtual environment within a physical store and practicing the presentation in that environment.

[1353] User operations

[1354] First, the user logs in to the terminal and enters the content of the presentation into the designated input form. For example, if they are describing a market launch plan for a new product, they enter the details. Then, by clicking the "Submit" button, the input content is sent to the server.

[1355] Creating documents using generative AI models

[1356] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. For example, based on input about a new product's market launch plan, it generates a title slide, product features, market analysis, launch schedule, and conclusion slides. The materials are saved in PDF or PPT format and sent to the device.

[1357] View and modify materials

[1358] The terminal displays the generated material to the user, who can check it and make corrections or additions as necessary.

[1359] Presentation skills evaluation

[1360] Next, the user practices the presentation, records the video, and uploads it to the system. The device then sends the recorded video file to the server. After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. Specifically, evaluations are made on points such as "speaking too fast" and "unfocused eye contact," and feedback is generated based on the results. This feedback is sent to the device and displayed to the user. The user can use this information to improve their presentation skills.

[1361] Synthetic speech demonstration

[1362] Additionally, users upload their presentation materials and a sample of their own voice to the system. The server uses a generative AI model to generate a synthetic voice that resembles the user's voice. This voice is then used to create a demonstration voice using the presentation materials and sent to the device. This allows users to use the synthesized voice as a model to improve their presentation content and speaking style.

[1363] Virtual practice

[1364] The user selects the VR practice mode and sets the desired scene. For example, they can select a virtual environment such as a large hall, a conference room, or a brick-and-mortar store. The scene setting information is sent to the server, which then generates the specified virtual environment. The virtual environment data is then sent to the device and VR goggles. The user then puts on the VR goggles and practices their presentation in the virtual environment. This allows the user to gain experience in a situation that is close to an actual presentation scene.

[1365] Usage example

[1366] For example, if a store staff member were to prepare a presentation document that briefly explains the features of a new product, an "eco vacuum cleaner," they could use the system based on this invention. By inputting the following prompt sentence into the generative AI model, the document would be quickly created.

[1367] Example prompt sentence:

[1368] Create a sales presentation for an eco-friendly vacuum cleaner.

[1369] 1. Title Slide

[1370] 2. Product Features

[1371] 3. Market analysis

[1372] 4. Implementation Schedule

[1373] 5. Conclusion

[1374] In this way, by utilizing the system of the present invention, it is possible to quickly create presentation materials, practice and evaluate them effectively, and improve presentation skills in a physical store.

[1375] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1376] Step 1:

[1377] The user logs in to the terminal and enters the content of the presentation into the specified input form. The input form contains fields for describing details such as the presentation theme, purpose, and target audience, and the input is sent to the server by pressing the "Send" button. The input is in text format.

[1378] Step 2:

[1379] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. Specific prompts are passed to the generative AI model for analysis. For example, based on a new product's market launch plan, a title slide, product features, market analysis, launch schedule, and conclusions are generated. The generated slides are saved in PDF or PPT format and sent to the device.

[1380] Step 3:

[1381] The device displays the generated presentation materials to the user. The user can review the content based on these materials and make corrections or additions as necessary. The corrected data is also sent to the server, where it can be processed again by the generative AI model.

[1382] Step 4:

[1383] A user practices a presentation and records a video of it. Once the recording is complete, the user uploads the video file to the system. The device then sends the recorded video file to the server. The video file is in a common video format (e.g., MP4).

[1384] Step 5:

[1385] The server analyzes the received video file using audio and image analysis. Audio analysis evaluates voice tone and speaking speed, while image analysis evaluates eye movements and posture. These evaluation points are extracted and specific feedback (e.g., "You speak too fast" or "Your gaze is not focused") is generated. Based on this feedback, areas for improvement and correction are extracted and sent to the device.

[1386] Step 6:

[1387] The device displays feedback to the user, who can then take action to improve their presentation skills.

[1388] Step 7:

[1389] Users upload their created presentation materials and a sample of their own voice to the system. The server then uses a generative AI model to generate synthetic speech based on the user's voice sample. This synthetic speech is then used to demonstrate the presentation materials, providing the user with a model presentation to use.

[1390] Step 8:

[1391] The device plays a demonstration of the generated synthesized voice to the user, who can use this as a reference to further improve the content and speaking style of their presentation.

[1392] Step 9:

[1393] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, or a brick-and-mortar store). This scene information is sent to the server, which then generates the specified virtual environment.

[1394] Step 10:

[1395] The generated virtual environment data is sent to the VR goggles via the terminal. The user can then put on the VR goggles and practice their presentation in the virtual environment, allowing them to gain experience in a situation that is close to a real presentation.

[1396] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1397] The system of the present invention includes a means for a user to input the content of a presentation and automatically generate presentation materials using a generative AI model, a means for displaying the generated presentation materials, a means for analyzing a video of a practice presentation and providing feedback, a means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, a means for practicing the presentation in a scene set up using virtual reality, and a means for analyzing emotions by combining an emotion engine.

[1398] Document creation function

[1399] User Action:

[1400] The user enters the content of the presentation into a dedicated input form. For example, they can enter content such as "Marketing strategy for the next product." Once the input is complete, they click the "Submit" button.

[1401] The device:

[1402] The presentation content sent by the user is sent to the server. The input content is properly formatted.

[1403] The server:

[1404] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials, such as a title slide, marketing strategy details, market analysis, action plan, and conclusion slides.

[1405] The device:

[1406] The generated presentation materials are received in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[1407] Evaluation function

[1408] User Action:

[1409] Users record videos of their practice presentations and upload them to the system.

[1410] The device:

[1411] The recorded video file is sent to the server.

[1412] The server:

[1413] The received video is analyzed. Specifically, audio analysis, image analysis, and an emotion engine are used to analyze the user's emotions.

[1414] The server:

[1415] Based on the analysis results, the system evaluates the presentation's delivery style, tone of voice, speed, eye movements, posture, and even emotional aspects, and generates feedback, including specific suggestions for improvement such as "adjust your speaking speed" and "maintain a consistent gaze."

[1416] The device:

[1417] Feedback is received and displayed to the user, who can use it to improve their next practice.

[1418] Demonstration function

[1419] User Action:

[1420] Users upload their presentation materials and a sample of their voice.

[1421] The device:

[1422] These data are sent to the server.

[1423] The server:

[1424] Based on the presentation materials and audio samples, a generative AI model is used to generate a synthetic voice that resembles the user's voice. This synthetic voice is then used to generate a demonstration audio file using the presentation materials.

[1425] The device:

[1426] The system receives a demonstration audio file and plays it for the user, who can use it as a model to improve their presentation content and speaking style.

[1427] Virtual practice function for each scene

[1428] User Action:

[1429] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[1430] The device:

[1431] Send scene setting information to the server.

[1432] The server:

[1433] The specified virtual environment is generated based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[1434] The device:

[1435] The received VR environment data is transferred to the VR goggles.

[1436] User Action:

[1437] Users put on VR goggles and practice their presentations in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[1438] The role of the emotional engine

[1439] The server:

[1440] The server uses an emotion engine to collect emotional data from the user's facial expressions, tone of voice, and speaking style during presentation practice videos and real-time VR practice sessions, detecting emotions such as tension, joy, and impatience.

[1441] The server:

[1442] The emotion engine analyzes the detected emotional data and generates feedback based on it, including specific advice such as "You seem nervous, so try slowing down your speech" or "Speak more emotively."

[1443] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[1444] The processing flow will be explained below.

[1445] Document creation function

[1446] Processing Steps:

[1447] Step 1:

[1448] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[1449] Step 2:

[1450] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[1451] Step 3:

[1452] The device formats the presentation content it receives and sends it to the server.

[1453] Step 4:

[1454] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[1455] Step 5:

[1456] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[1457] Step 6:

[1458] Save the generated presentation materials in PDF or PPT format and send them to your device.

[1459] Step 7:

[1460] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[1461] Evaluation function

[1462] Processing Steps:

[1463] Step 1:

[1464] A user records a video of their practice presentation, which includes the user giving the presentation.

[1465] Step 2:

[1466] Users upload recorded videos to the system.

[1467] Step 3:

[1468] The device sends the uploaded video file to the server.

[1469] Step 4:

[1470] The server analyzes the received video. Specifically, it performs audio and image analysis, and then extracts emotional data using an emotion engine.

[1471] Step 5:

[1472] The server evaluates speaking style, tone of voice, speed, eye movements, posture, and emotions, and extracts evaluation points, such as "speaking too fast," "unfocused eye contact," and "appears nervous."

[1473] Step 6:

[1474] The server generates feedback based on the analysis results. The feedback includes suggestions for improving speaking style and posture, as well as emotional advice. For example, "You seem nervous, so try taking a deep breath" or "Try to lower your voice tone and speak calmly."

[1475] Step 7:

[1476] The server generates feedback and sends it to the device.

[1477] Step 8:

[1478] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[1479] Demonstration function

[1480] Processing Steps:

[1481] Step 1:

[1482] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[1483] Step 2:

[1484] The device sends the uploaded presentation materials and voice sample to the server.

[1485] Step 3:

[1486] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[1487] Step 4:

[1488] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[1489] Step 5:

[1490] The server transmits the generated demonstration audio file to the terminal.

[1491] Step 6:

[1492] The device plays the demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[1493] Virtual practice function for each scene

[1494] Processing Steps:

[1495] Step 1:

[1496] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[1497] Step 2:

[1498] The device sends the scene setting information to the server.

[1499] Step 3:

[1500] The server generates a specified virtual environment based on the scene information received.

[1501] Step 4:

[1502] The VR environment data generated by the server is sent to the terminal and VR goggles.

[1503] Step 5:

[1504] The terminal transfers the received VR environment data to the VR goggles.

[1505] Step 6:

[1506] The user puts on VR goggles and practices their presentation in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[1507] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[1508] Example 2

[1509] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1510] Traditional methods for creating, practicing, and evaluating presentation materials require a great deal of time and effort, making it difficult to improve the quality of presentations. In particular, in order to receive feedback on one's own performance, one must request evaluations from others, making it difficult to receive objective and continuous feedback. Furthermore, the content and structure of presentation materials must be created manually, making it difficult to generate materials quickly and effectively.

[1511] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1512] In this invention, the server includes: means for a user to input presentation content; means for using a generative AI model to automatically generate presentation materials based on the input content; means for displaying the generated presentation materials; means for analyzing a video of a practice presentation and providing feedback based on the results using audio analysis, image analysis, and the generative AI model; means for demonstrating a model presentation using a synthesized voice that resembles the user's voice; means for practicing the presentation in a scene set using virtual reality; and means for analyzing emotional data during the presentation using an emotion engine and providing feedback. This allows users to quickly and automatically create presentation materials and receive multifaceted feedback to improve the quality of their presentations.

[1513] "Presentation content" refers to the information and data that a user inputs to create presentation materials.

[1514] A "generative AI model" is an artificial intelligence algorithm that automatically creates presentation materials based on input content.

[1515] "Presentation Materials" means the content of a presentation in slide or other format, typically generated as a PDF or PPT file.

[1516] "Voice analysis" is a technology that analyzes recorded voice data and evaluates the tone, speed, quality of the voice, etc.

[1517] "Image analysis" is a technology that analyzes recorded video data and evaluates eye movements, facial expressions, posture, etc.

[1518] "Feedback" is an evaluation of the content and performance of a presentation and specific advice for improvement.

[1519] "Synthetic voice" is an artificial voice that resembles the user's voice and is generated based on a user's voice sample.

[1520] "Virtual reality" refers to a virtual digital environment generated using computer technology in which users can interact.

[1521] The "emotion engine" is a technology that analyzes a user's emotions from facial expressions, tone of voice, etc., and detects specific emotional states.

[1522] A "scene" refers to a specific environment or location within virtual reality where a presentation takes place.

[1523] MODE FOR CARRYING OUT THE INVENTION

[1524] The system of the present invention automatically generates presentation materials based on the content entered by the user in order to improve the quality of presentations, and provides multifunctional support for supporting presentation practice.

[1525] Document creation function

[1526] User operations

[1527] The user enters the content of the presentation into a dedicated input form, for example, a specific content such as "Marketing strategy for the next product," and clicks the submit button.

[1528] Device operation

[1529] The device formats the input presentation content appropriately and sends it to the server. The data sent is in text format and is encoded.

[1530] Server Operations

[1531] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. Specifically, slides are generated, including a title slide, details of the marketing strategy, market analysis, an action plan, and a conclusion. Various natural language processing technologies are used in this generative AI model.

[1532] Device operation

[1533] The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[1534] Prompt Sentence Examples

[1535] "Please create a presentation about the marketing strategy for our next product."

[1536] Evaluation function

[1537] User operations

[1538] The user records a video of the presentation practice and uploads the video file to the system.

[1539] Device operation

[1540] The device sends the recorded video file to the server, where the file format and size are checked.

[1541] Server Operations

[1542] The server analyzes the received video using audio analysis, image analysis, and an emotion engine, and analyzes voice tone, speed, eye movement, posture, emotion, etc.

[1543] Server Operations

[1544] Feedback is generated based on the analysis results, including suggestions for improvement such as "adjust speaking speed" and "maintain consistent eye contact."

[1545] Device operation

[1546] The feedback is sent to the device and displayed to the user, who can use it to improve their next practice.

[1547] Prompt Sentence Examples

[1548] "Record yourself practicing your presentation and provide feedback."

[1549] Demonstration function

[1550] User operations

[1551] Users upload their own presentation materials and voice samples to the system.

[1552] Device operation

[1553] Your presentation and audio samples are sent to the server, where they are checked for correct file format and size.

[1554] Server Operations

[1555] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the uploaded voice sample, and then uses the generated synthetic voice to create a demonstration audio file of the presentation material.

[1556] Device operation

[1557] The demonstration audio file is sent to the terminal and played back for the user, who can use the audio file as a model to improve the content and speaking style of their presentation.

[1558] Prompt Sentence Examples

[1559] "Please convert your presentation materials into audio files and play them as a model."

[1560] Virtual practice function for each scene

[1561] User operations

[1562] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). Scenes can be selected from the system's catalog.

[1563] Device operation

[1564] Scene setting information is sent to the server, including the selected scene's ID and detailed parameters.

[1565] Server Operations

[1566] The server generates the specified virtual environment based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[1567] Device operation

[1568] The terminal transfers the received VR environment data to the VR goggles.

[1569] User operations

[1570] Users put on VR goggles and practice their presentation in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data.

[1571] Prompt Sentence Examples

[1572] "I want to practice my presentation in a virtual conference room environment."

[1573] By combining these functions, users can quickly create presentation materials and practice them with detailed feedback. Furthermore, by using an emotion engine, it is possible to analyze emotions such as tension and joy and provide more effective feedback.

[1574] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1575] Document creation function

[1576] Step 1:

[1577] User actions

[1578] The user enters the content of the presentation into a special input form and clicks the send button. The input is specific content such as "marketing strategy for the next product." The output is the content of the form being sent to the terminal as data.

[1579] Step 2:

[1580] Processing performed by the device

[1581] The device properly formats the presentation content received from the user and sends it to the server. The input is the text data of the presentation content sent by the user, and the output is sending the formatted data to the server. Specifically, the data is formatted, encoded, and sent to the server according to the communication protocol.

[1582] Step 3:

[1583] Processing performed by the server

[1584] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. The input is the formatted presentation content, and the output is the generated presentation material. Specific operations include text analysis, determining the slide structure and layout, and generating automatically inserted images and graphs.

[1585] Step 4:

[1586] Processing performed by the device

[1587] The terminal receives the generated presentation materials in PDF or PPT format and displays them to the user. The input is the presentation material file sent from the server, and the output is to display that file on the user's device. Specific operations include downloading the file, displaying it in a viewer, and accepting edits.

[1588] Evaluation function

[1589] Step 1:

[1590] User actions

[1591] A user records a video of a practice presentation and uploads the video file to the system. The input is the recorded video file, and the output is the uploaded video data.

[1592] Step 2:

[1593] Processing performed by the device

[1594] The device sends the recorded video file to the server. The input is the video file uploaded by the user, and the output is to send this file to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[1595] Step 3:

[1596] Processing performed by the server

[1597] The server uses audio analysis, image analysis, and an emotion engine to analyze the received video. The input is the received video file, and the output is the analysis results and feedback. Specific operations include analyzing voice tone and speed, analyzing eye movements and posture, and detecting emotional states.

[1598] Step 4:

[1599] Processing performed by the server

[1600] Feedback is generated based on the analysis results. The input is the analysis results, and the output is the generated feedback. Specific operations include extracting areas for improvement based on the analysis data and generating specific feedback statements.

[1601] Step 5:

[1602] Processing performed by the device

[1603] The terminal receives the feedback data and displays it to the user. The input is the feedback data sent from the server, and the output is to show this data to the user. Specific operations include displaying feedback text, hints, and operation guides for the user's next practice.

[1604] Demonstration function

[1605] Step 1:

[1606] User actions

[1607] Users upload their created presentation materials and their own voice samples to the system. The input is the presentation material file and the voice sample file, and the output is the transfer of these data to the terminal.

[1608] Step 2:

[1609] Processing performed by the device

[1610] The terminal sends presentation materials and audio samples to the server. The input is the file uploaded by the user, and the output is to send these files to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[1611] Step 3:

[1612] Processing performed by the server

[1613] The server uses a generative AI model to generate synthetic speech that resembles the user's voice based on the uploaded voice sample. This synthetic speech is then used to create a demonstration audio file of the presentation materials. The input is the voice sample and the presentation materials, and the output is the synthesized demonstration audio file. Specific operations include speech synthesis, audio file generation, and reading of the presentation materials.

[1614] Step 4:

[1615] Processing performed by the device

[1616] Receives a demonstration audio file and plays it for the user. The input is the audio file sent from the server, and the output is the playback of the audio file. Specific operations include downloading the file, playing it in an audio player, and accepting user operations.

[1617] Virtual practice function for each scene

[1618] Step 1:

[1619] User actions

[1620] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The input is the scene setting information, and the output is the set information being sent to the device.

[1621] Step 2:

[1622] Processing performed by the device

[1623] Sends scene setting information to the server. The input is the scene information set by the user, and the output is to send this information to the server. Specific operations include formatting the setting information and communicating it according to the transmission protocol.

[1624] Step 3:

[1625] Processing performed by the server

[1626] The server generates a virtual environment based on the received scene information. The input is the received scene information, and the output is the generated virtual environment data. Specific operations include 3D modeling of the virtual environment and scenario setting.

[1627] Step 4:

[1628] Processing performed by the device

[1629] The received VR environment data is transmitted to the VR goggles. The input is the virtual environment data sent from the server, and the output is the data transmitted to the VR goggles. The specific operations include decoding the data and transmitting it to the device.

[1630] Step 5:

[1631] User actions

[1632] Users put on VR goggles and practice their presentations in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data. The input is the actual presentation's movements and emotional data, and the output is real-time feedback.

[1633] The above are the processing steps in the system of the present invention, which allows users to achieve comprehensive improvement in their presentation skills.

[1634] (Application example 2)

[1635] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1636] Conventional presentation practice support systems incorporate functions for automatically generating presentation materials and providing feedback on practice, but they are not adequately suited to real-world work environments, particularly in factory environments. Furthermore, because they do not provide real-time feedback based on emotional data, it is difficult to effectively alleviate issues such as tension and impatience. This results in problems such as reduced user work efficiency and reduced teaching effectiveness.

[1637] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1638] In this invention, the server includes means for a user to input presentation content, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for analyzing a video of a practice presentation and providing feedback based on the results, means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, means for practicing the presentation in a scene set using virtual reality, means for analyzing a recorded video of the practice presentation, collecting emotional data and providing feedback, and means for providing work guidance in a factory using virtual reality and robots. This allows users to practice in a simulation that resembles an actual work environment, and by receiving feedback based on the emotional data, the quality of their presentations can be improved.

[1639] "User" refers to a person who uses the presentation system to create presentation materials, practice them, and receive feedback.

[1640] "Presentation content" refers to the information and outline that a user enters when creating presentation materials.

[1641] "Auto-generation" refers to the process of automatically generating presentation materials using machine learning models and algorithms based on input information and prompts.

[1642] "Presentation materials" refers to materials such as slides and documents used to effectively communicate the content of a presentation.

[1643] "Display means" refers to a device or method for visually presenting the generated presentation materials to the user.

[1644] "Feedback" means information or comments that provide improvements or evaluations based on the results of your practice presentation.

[1645] "Synthetic voice" refers to a voice that resembles a human voice and is generated using voice synthesis technology.

[1646] "Virtual reality" means a virtual environment created using computer graphics and other technologies, in which users can have a realistic experience.

[1647] "Emotional data" refers to information about the user's emotional state analyzed from facial expressions, voice, etc.

[1648] "Work instruction" means the process of providing instruction and training regarding work tasks and procedures within a factory.

[1649] A "robot" refers to an automated machine that performs programmed actions and provides information and work instructions to users.

[1650] The system of the present invention allows users to input presentation content, automatically generate presentation materials using a generative AI model, display and evaluate the materials, and practice the presentation in a virtual reality environment. To effectively implement the system of the present invention, the following hardware and software are used:

[1651] Hardware and software used

[1652] 1. Hardware

[1653] Server: High-performance data processing server

[1654] Device: A computer or tablet that a user operates.

[1655] VR system: Virtual reality goggles such as Oculus Quest 2

[1656] Robot: A multi-functional robot that provides guidance and feedback within the factory

[1657] 2. Software

[1658] Generative AI model: OpenAI GPT-4

[1659] Sentiment analysis model: Custom trained model using TensorFlow and Keras

[1660] Video analysis software: OpenCV and MoviePy

[1661] Voice analysis software: Librosa

[1662] Data Transfer and Management System: Cloud Database

[1663] System processing overview

[1664] First, the user inputs the content of the presentation using a dedicated terminal. For example, they input a prompt such as "Operating procedure for new equipment." Once the input is complete, the terminal sends the content to the server.

[1665] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (OpenAI GPT-4). These materials include a title slide, operating procedures, safety precautions, etc. The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user.

[1666] Next, the user records a video of their practice presentation and uploads it to the server via their device. The server receives the video, performs audio analysis (Librosa) and image analysis (OpenCV), and extracts emotional data using emotion analysis models (TensorFlow and Keras). This allows the server to determine emotional states such as tension or joy, and generates feedback based on the results.

[1667] Furthermore, a synthesized voice generated from a sample of the user's voice is created to demonstrate the presentation materials and provided to the user, allowing the user to learn from the model presentation.

[1668] Additionally, using a virtual reality system (Oculus Quest 2), users can practice their presentations in a designated scene (e.g., a specific location in a factory). The server generates this virtual environment and provides real-time feedback based on emotional data.

[1669] Finally, the robot will provide work instructions within the factory, explaining specific operating procedures and providing real-time guidance based on presentation materials and feedback.

[1670] Specific examples

[1671] For example, when leading a presentation on new safety procedures in a factory, a worker inputs the prompt phrase "new safety procedure." The generative AI model automatically creates presentation materials and displays them on the device. The worker can then give a presentation based on these materials and automatically receive feedback from the recorded video.

[1672] Prompt Sentence Examples

[1673] Prompt: "Operating procedures for new equipment"

[1674] Generated presentation contents:

[1675] Title: New equipment operating procedures

[1676] Slide 1: Facility Overview

[1677] Slide 2: Operational Procedure 1

[1678] Slide 3: Operational Procedure 2

[1679] Slide 4: Safety Precautions

[1680] Slide 5: Questions and Answers

[1681] In this way, the system of the present invention can comprehensively improve users' presentation skills and work efficiency through the rapid creation of presentation materials, evaluation of practice, provision of advanced feedback using emotional data, and practical instruction using virtual reality and robots.

[1682] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1683] Step 1:

[1684] The user inputs the content of the presentation. The user enters a prompt sentence into a dedicated input form and clicks the submit button. For example, the user enters "Operating procedures for new equipment." This input becomes the initial data for the system.

[1685] Step 2:

[1686] The device sends the input presentation content to the server. The device then formats the input content and sends it to the server in a format that is easy for the generative AI model to analyze. This is where data is sent and received.

[1687] Step 3:

[1688] Based on the presentation content received by the server, a generative AI model (OpenAI GPT-4) is used to automatically generate presentation materials. The generative AI model analyzes the input prompts and creates slides (PDF or PPT format) to present relevant information, such as a title slide, operating procedures, and safety precautions.

[1689] Step 4:

[1690] The server sends the generated presentation materials to the terminal. The generated materials are transferred to the terminal in the appropriate format (PDF or PPT) and displayed to the user. The user can check the materials and make corrections as necessary.

[1691] Step 5:

[1692] The user records a video of their practice presentation and uploads it to the server via their device. The recorded video is sent to the server and becomes the data for the next analysis step.

[1693] Step 6:

[1694] The server analyzes the received video. Specifically, audio analysis (Librosa) and image analysis (OpenCV) are performed, and emotional data is extracted using an emotion analysis model (TensorFlow and Keras). For example, MFCC features are extracted to determine the emotional state. This allows emotions such as tension and joy to be detected.

[1695] Step 7:

[1696] The server generates feedback based on the analysis results, including adjustments to speaking speed and improvements to eye contact. The server creates the feedback information and notifies the device.

[1697] Step 8:

[1698] The device displays the received feedback to the user, who can review the feedback and receive guidance on how to improve the presentation's delivery and content.

[1699] Step 9:

[1700] The user uploads a sample of their voice to the server, which then serves as input for generating the synthesized voice.

[1701] Step 10:

[1702] The server generates synthetic speech. Using a generative AI model, it generates synthetic speech that resembles the user's voice. Based on this synthetic speech, a model presentation audio file is created.

[1703] Step 11:

[1704] The server sends the synthesized voice file to the terminal and plays it back to the user, who can use it to practice their presentation.

[1705] Step 12:

[1706] Users can use the virtual reality system to practice presentations in a specified scene. For example, they can wear an Oculus Quest 2 and simulate a virtual conference room or factory.

[1707] Step 13:

[1708] The server collects practice data in the virtual reality environment in real time and performs emotional analysis. Based on the user's movements, facial expressions, and tone of voice, real-time feedback is generated and presented to the user in the virtual reality environment.

[1709] Step 14:

[1710] The robot provides work instruction within the factory. Based on the generated presentation materials and feedback, the robot explains specific operating procedures and provides real-time instruction. By receiving instruction from the robot, users can improve their practical skills.

[1711] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1712] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1713] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1714] [Fourth embodiment]

[1715] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1716] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1717] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1718] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1719] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1720] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1721] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1722] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1723] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1724] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1725] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1726] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1727] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1728] The system of the present invention is designed to assist users who are not good at or have little experience creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[1729] Document creation function

[1730] User Action:

[1731] The user enters the content of the presentation into the designated input form. For example, if the user wants to present about the market launch plan of a new product, the user writes the content in detail in the input form. Once the input is complete, the user clicks the "Submit" button.

[1732] The device:

[1733] The input presentation content is sent to the server, where it is properly formatted.

[1734] The server:

[1735] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. For example, it generates a title slide, product features slide, market analysis slide, introduction schedule slide, and conclusion slide based on the input topic, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the device.

[1736] The device:

[1737] The received presentation materials are displayed to the user, who can then review them and make corrections or additions as necessary.

[1738] Evaluation function

[1739] User Action:

[1740] Users record videos of their practice presentations and upload them to the system.

[1741] The device:

[1742] The recorded video file is sent to the server.

[1743] The server:

[1744] The server that receives the video performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, specific evaluations are made such as "speaking too fast" or "unfocused gaze," and feedback is generated based on the results. This feedback is then sent to the device.

[1745] The device:

[1746] The received feedback is displayed to the user, allowing them to see areas for improvement and correction, and to adjust their next practice based on this.

[1747] Demonstration function

[1748] User Action:

[1749] Users upload the presentation materials they have created and a sample of their own voice (audio file) to the system.

[1750] The device:

[1751] Send the uploaded data to the server.

[1752] The server:

[1753] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice samples. The server then uses the voice to demonstrate the presentation materials and generate a synthetic voice presentation. The generated audio file is then sent to the device.

[1754] The device:

[1755] The demonstration audio is played for users, who can use it as a model to improve their presentation content and speaking style.

[1756] Virtual practice function for each scene

[1757] User Action:

[1758] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[1759] The device:

[1760] Send scene setting information to the server.

[1761] The server:

[1762] The server receives the scene information and generates the specified virtual environment, and the virtual environment data is sent to the device and the VR goggles.

[1763] The device:

[1764] The received VR environment data is transferred to the VR goggles.

[1765] User Action:

[1766] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual large hall, they can gain experience in a situation similar to a real presentation.

[1767] As described above, the system of the present invention provides consistent support from the rapid creation of presentation materials to effective practice methods, thereby improving the presentation skills of users.

[1768] The processing flow will be explained below.

[1769] Document creation function

[1770] Processing Steps:

[1771] Step 1:

[1772] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[1773] Step 2:

[1774] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[1775] Step 3:

[1776] The device formats the presentation content it receives and sends it to the server.

[1777] Step 4:

[1778] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[1779] Step 5:

[1780] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[1781] Step 6:

[1782] Save the generated presentation materials in PDF or PPT format and send them to your device.

[1783] Step 7:

[1784] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[1785] Evaluation function

[1786] Processing Steps:

[1787] Step 1:

[1788] A user records a video of their practice presentation, which includes the user giving the presentation.

[1789] Step 2:

[1790] Users upload recorded videos to the system.

[1791] Step 3:

[1792] The device sends the uploaded video file to the server.

[1793] Step 4:

[1794] The server analyzes the received video, specifically, analyzing the audio and images.

[1795] Step 5:

[1796] The server extracts evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the speaker's speaking speed or their gaze is not focused.

[1797] Step 6:

[1798] The server generates feedback based on the analysis results, including suggestions for improvement and correction.

[1799] Step 7:

[1800] The server generates feedback and sends it to the device.

[1801] Step 8:

[1802] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[1803] Demonstration function

[1804] Processing Steps:

[1805] Step 1:

[1806] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[1807] Step 2:

[1808] The device sends the uploaded presentation materials and voice sample to the server.

[1809] Step 3:

[1810] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[1811] Step 4:

[1812] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[1813] Step 5:

[1814] The server transmits the generated demonstration audio file to the terminal.

[1815] Step 6:

[1816] The device displays a demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[1817] Virtual practice function for each scene

[1818] Processing Steps:

[1819] Step 1:

[1820] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[1821] Step 2:

[1822] The device sends the scene setting information to the server.

[1823] Step 3:

[1824] The server generates a specified virtual environment based on the scene information received.

[1825] Step 4:

[1826] The VR environment data generated by the server is sent to the terminal and VR goggles.

[1827] Step 5:

[1828] The terminal transfers the received VR environment data to the VR goggles.

[1829] Step 6:

[1830] Users put on VR goggles and practice their presentations in a virtual environment. For example, by practicing giving a presentation in a virtual hall, they can gain experience in a situation that is close to a real presentation.

[1831] Example 1

[1832] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1833] Creating and delivering presentations is a difficult, time-consuming and laborious task for many users, placing a significant burden on beginners and those with little experience. This issue extends beyond support for creating presentation materials to the improvement of actual presentation skills. Conventional methods require time to create presentation materials and lack efficient practice methods, making it difficult to effectively improve presentation skills in a short period of time.

[1834] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1835] In this invention, the server includes means for analyzing the presentation content and automatically generating presentation materials using a generative AI model, means for saving the generated presentation materials in PDF or PPT format and sending them to a terminal, and means for analyzing a presentation practice video and providing feedback based on the results. This allows for the rapid generation of presentation materials based on the content entered by the user, and further analysis of the practice video and provision of feedback, enabling effective improvement of presentation skills in a short period of time.

[1836] "User" refers to a person who uses the system to create or practice presentation materials.

[1837] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[1838] "Server" refers to the computer system that processes data, runs the generative AI model, stores and transmits generated materials, and provides analysis and feedback.

[1839] A "generative AI model" refers to an artificial intelligence model that automatically generates presentation materials based on input text data.

[1840] "Presentation materials" refers to slides, documents, etc. that visually display the contents of the presentation.

[1841] "Input form" refers to the interface that allows users to input the content of the presentation.

[1842] "Feedback" refers to evaluations and suggestions for improvement provided to users that are generated based on the results of analyzing the presentation practice video.

[1843] "Synthetic voice" refers to artificial voice data generated to resemble the user's voice.

[1844] "Virtual reality" refers to a simulated environment generated using computer technology.

[1845] "Simulation" refers to the near-realistic reproduction of a particular scenario or environment.

[1846] "Prompts" refer to guidelines or instructions used by generative AI models to generate presentation materials.

[1847] The system of the present invention is designed to assist users who are not good at creating and giving presentation materials, and uses a generative AI model to significantly reduce the time it takes to create presentation materials and improve presentation skills. This system is mainly implemented by a server, a terminal, and a user.

[1848] First, the user enters the content of the presentation into a specified input form. For example, if a user wants to present a new product's market launch plan, they enter the details and click the "Submit" button. The device then converts the input presentation content into an appropriate format and sends it to the server using a protocol such as an HTTP POST request.

[1849] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model (e.g., OpenAI's GPT-4). Specifically, based on the input theme, it generates a title slide, product features, market analysis, introduction schedule, and conclusion slides, inserting appropriate diagrams and graphs into each. The generated slides are saved in PDF or PPT format and sent to the terminal using file transfer protocols (FTP or HTTP).

[1850] The device displays the received presentation materials to the user, who can then review them and make corrections or additions as necessary.

[1851] Example prompt sentence:

[1852] "Please create a presentation about your new product launch plan. Key points should include product features, market analysis, launch schedule, and expected results."

[1853] Next, the user can record a video of the practice presentation and upload it to the system. The device sends the recorded video file to the server using a protocol such as an HTTP POST request.

[1854] After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. For example, it may evaluate the user's speaking speed or their eye gaze is not focused, and generate detailed feedback based on the results. The generated feedback is sent to the device in a text message format (e.g., JSON).

[1855] The device displays the received feedback to the user, allowing the user to see areas for improvement or correction.

[1856] In addition, users can upload their created presentation materials and a sample of their own voice (audio file) to the system. The device sends the data to the server, which uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the presentation materials and voice sample. The presentation materials are then demonstrated using this synthetic voice, and a synthetic voice presentation is generated. The generated presentation is then sent to the device.

[1857] The device plays back the received synthesized speech presentation, allowing the user to use it as a model to improve their presentation content and speaking style.

[1858] Finally, the user can select the VR practice mode and set the desired scene (e.g., a large hall or a conference room). The device sends the scene setting information to the server, which then generates the specified virtual environment. This virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine) and sent to the device and VR goggles.

[1859] The device then transfers the received VR environment data to the VR goggles, where the user can practice their presentation in the virtual environment. For example, by practicing a presentation in a virtual large hall, users can gain experience in a situation similar to a real presentation.

[1860] As described above, the system of the present invention consistently supports everything from the rapid creation of presentation materials to effective practice methods, thereby improving the user's presentation skills.

[1861] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1862] Step 1:

[1863] The user enters the details of the presentation in the designated input form. When the input is complete, the user clicks the "Send" button. The input is sent to the terminal as text data.

[1864] Step 2:

[1865] The terminal receives the text data entered by the user and formats it into an appropriate format (such as JSON). The formatted data is then sent to the server using an HTTP POST request. Specifically, the text data entered in the input form is converted into JSON format.

[1866] Step 3:

[1867] The server receives the JSON-formatted text data sent via an HTTP POST request, parses the received data, and extracts the theme and content of the presentation. During this analysis process, the analysis engine identifies keywords and context in the input data.

[1868] Step 4:

[1869] The server inputs the analyzed data into a generative AI model (e.g., GPT-4). The generative AI model automatically generates presentation materials based on the input data. The generative AI model generates slides containing content such as a title slide, product features, market analysis, introduction schedule, and conclusion.

[1870] Step 5:

[1871] The server saves the generated presentation slides in PDF or PPT format, and the saved files are sent to the terminal via HTTP or FTP. Specifically, the generated slides are converted to PDF format and saved in the specified directory.

[1872] Step 6:

[1873] The terminal receives the PDF or PPT file from the server and displays it to the user using a PDF reader or PPT viewer. The user can then review the document and make corrections or additions as necessary.

[1874] Step 7:

[1875] A user records a video of their practice presentation and uploads it to the system. The recorded video file is recognized by the device and sent to the server. Specifically, a video file is selected and uploaded to the server via an HTTP POST request.

[1876] Step 8:

[1877] The server receives the uploaded video file and performs audio and image analysis, extracting evaluation points such as speaking style, tone of voice, speed, eye movements, and posture.

[1878] Step 9:

[1879] The server generates feedback based on the analyzed evaluation points. The feedback is generated in text message format (e.g., JSON) and includes specific improvements and corrections. This feedback is sent to the device as an HTTP response.

[1880] Step 10:

[1881] The device analyzes the feedback received from the server and displays it to the user, who can then review the feedback and make improvements to their next practice session.

[1882] Step 11:

[1883] The user uploads the presentation materials they have created and a sample of their own voice (an audio file) to the system. The device sends the data to the server. Specifically, the presentation materials and audio file are selected and uploaded to the server via an HTTP POST request.

[1884] Step 12:

[1885] The server analyzes the received presentation materials and voice samples and uses a generative AI model to generate a synthetic voice that matches the content of the presentation materials.

[1886] Step 13:

[1887] The server then integrates the synthesized speech into the presentation materials to generate a live audio presentation, which is then saved as an audio file and sent to the device via HTTP or FTP.

[1888] Step 14:

[1889] The terminal plays the presentation with audio received from the server and displays it to the user, who can use it as a model to improve their own presentation skills.

[1890] Step 15:

[1891] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The device sends the setting information to the server. Specifically, the VR scene selection information is sent to the server in JSON format.

[1892] Step 16:

[1893] The server generates a virtual environment based on the VR scene information. The virtual environment is created using a 3D graphics engine (e.g., Unity, Unreal Engine).

[1894] Step 17:

[1895] The virtual environment data is sent from the server to the device and the VR goggles. The device then transfers the received VR environment data to the VR goggles. The user can then wear the VR goggles and practice their presentation in the virtual environment.

[1896] The above is the flow of program processing, including the specific operations performed at each step and an explanation of the inputs and outputs.

[1897] (Application example 1)

[1898] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1899] Creating presentation materials and practicing presentations is difficult and time-consuming for many users. Furthermore, in brick-and-mortar stores, staff are required to be able to give efficient presentations, such as when selling new products or introducing services, and the associated skills training is also important. Therefore, there is a need for a system that allows users to efficiently create presentation materials, practice, evaluate, and demonstrate them, and also improve their presentation skills in brick-and-mortar stores.

[1900] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1901] In this invention, the server includes a means for a user to input the content of a presentation, a means for automatically generating presentation materials based on the input content, a means for displaying the generated presentation materials, a means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, a means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and a means for generating a virtual environment within a physical store and practicing the presentation within that environment. This enables users to quickly create, evaluate, and receive feedback on their presentation materials, as well as provide effective training within the physical store.

[1902] "User" refers to an individual or organizational member who creates, evaluates, and practices presentation materials.

[1903] "Input means" refers to an interface or device that allows a user to provide specific content to a system.

[1904] "Automatic generation means" refers to software and algorithms for generating presentation materials based on specified inputs.

[1905] "Displaying means" refers to a display device or software that allows users to see the generated presentation materials and feedback.

[1906] "Video analysis means" means software and hardware for analyzing and evaluating the video and audio of a recorded presentation.

[1907] "Means for providing feedback" refers to interfaces and algorithms for providing users with suggestions for improvement and advice based on the analysis results.

[1908] "Means for demonstrating using synthetic speech" refers to technology that generates similar synthetic speech based on a user's voice sample and uses the speech sample to demonstrate the speech using presentation materials.

[1909] "Virtual reality methods" refers to the use of VR devices and software to create a virtual environment in which to practice a presentation.

[1910] "System" refers to the entire software and hardware collection that combines the above means to consistently support the creation, practice, and evaluation of presentation materials.

[1911] "Synthetic voice" refers to an artificial voice that is generated to resemble the user's voice.

[1912] "Virtual environment" refers to a computer-generated simulation that allows users to experience situations similar to those in the real world.

[1913] An "in-store" virtual environment refers to a virtual reality scene that mimics an actual in-store situation.

[1914] "Voice analysis" refers to the technology of analyzing recorded audio data to extract evaluation points such as tone of voice and speaking speed.

[1915] "Image analysis" refers to the technology of analyzing recorded video data to extract evaluation points such as eye movements and posture.

[1916] The system based on this invention basically comprises the following means: means for a user to input the content of the presentation, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for performing audio and image analysis based on the presentation video to evaluate the presentation skills of staff, means for generating synthetic voice and automatically performing a demonstration using the presentation materials, and means for generating a virtual environment within a physical store and practicing the presentation in that environment.

[1917] User operations

[1918] First, the user logs in to the terminal and enters the content of the presentation into the designated input form. For example, if they are describing a market launch plan for a new product, they enter the details. Then, by clicking the "Submit" button, the input content is sent to the server.

[1919] Creating documents using generative AI models

[1920] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. For example, based on input about a new product's market launch plan, it generates a title slide, product features, market analysis, launch schedule, and conclusion slides. The materials are saved in PDF or PPT format and sent to the device.

[1921] View and modify materials

[1922] The terminal displays the generated material to the user, who can check it and make corrections or additions as necessary.

[1923] Presentation skills evaluation

[1924] Next, the user practices the presentation, records the video, and uploads it to the system. The device then sends the recorded video file to the server. After receiving the video, the server performs audio and image analysis to extract evaluation points such as speaking style, tone of voice, speed, eye movement, and posture. Specifically, evaluations are made on points such as "speaking too fast" and "unfocused eye contact," and feedback is generated based on the results. This feedback is sent to the device and displayed to the user. The user can use this information to improve their presentation skills.

[1925] Synthetic speech demonstration

[1926] Additionally, users upload their presentation materials and a sample of their own voice to the system. The server uses a generative AI model to generate a synthetic voice that resembles the user's voice. This voice is then used to create a demonstration voice using the presentation materials and sent to the device. This allows users to use the synthesized voice as a model to improve their presentation content and speaking style.

[1927] Virtual practice

[1928] The user selects the VR practice mode and sets the desired scene. For example, they can select a virtual environment such as a large hall, a conference room, or a brick-and-mortar store. The scene setting information is sent to the server, which then generates the specified virtual environment. The virtual environment data is then sent to the device and VR goggles. The user then puts on the VR goggles and practices their presentation in the virtual environment. This allows the user to gain experience in a situation that is close to an actual presentation scene.

[1929] Usage example

[1930] For example, if a store staff member were to prepare a presentation document that briefly explains the features of a new product, an "eco vacuum cleaner," they could use the system based on this invention. By inputting the following prompt sentence into the generative AI model, the document would be quickly created.

[1931] Example prompt sentence:

[1932] Create a sales presentation for an eco-friendly vacuum cleaner.

[1933] 1. Title Slide

[1934] 2. Product Features

[1935] 3. Market analysis

[1936] 4. Implementation Schedule

[1937] 5. Conclusion

[1938] In this way, by utilizing the system of the present invention, it is possible to quickly create presentation materials, practice and evaluate them effectively, and improve presentation skills in a physical store.

[1939] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1940] Step 1:

[1941] The user logs in to the terminal and enters the content of the presentation into the specified input form. The input form contains fields for describing details such as the presentation theme, purpose, and target audience, and the input is sent to the server by pressing the "Send" button. The input is in text format.

[1942] Step 2:

[1943] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials. Specific prompts are passed to the generative AI model for analysis. For example, based on a new product's market launch plan, a title slide, product features, market analysis, launch schedule, and conclusions are generated. The generated slides are saved in PDF or PPT format and sent to the device.

[1944] Step 3:

[1945] The device displays the generated presentation materials to the user. The user can review the content based on these materials and make corrections or additions as necessary. The corrected data is also sent to the server, where it can be processed again by the generative AI model.

[1946] Step 4:

[1947] A user practices a presentation and records a video of it. Once the recording is complete, the user uploads the video file to the system. The device then sends the recorded video file to the server. The video file is in a common video format (e.g., MP4).

[1948] Step 5:

[1949] The server analyzes the received video file using audio and image analysis. Audio analysis evaluates voice tone and speaking speed, while image analysis evaluates eye movements and posture. These evaluation points are extracted and specific feedback (e.g., "You speak too fast" or "Your gaze is not focused") is generated. Based on this feedback, areas for improvement and correction are extracted and sent to the device.

[1950] Step 6:

[1951] The device displays feedback to the user, who can then take action to improve their presentation skills.

[1952] Step 7:

[1953] Users upload their created presentation materials and a sample of their own voice to the system. The server then uses a generative AI model to generate synthetic speech based on the user's voice sample. This synthetic speech is then used to demonstrate the presentation materials, providing the user with a model presentation to use.

[1954] Step 8:

[1955] The device plays a demonstration of the generated synthesized voice to the user, who can use this as a reference to further improve the content and speaking style of their presentation.

[1956] Step 9:

[1957] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, or a brick-and-mortar store). This scene information is sent to the server, which then generates the specified virtual environment.

[1958] Step 10:

[1959] The generated virtual environment data is sent to the VR goggles via the terminal. The user can then put on the VR goggles and practice their presentation in the virtual environment, allowing them to gain experience in a situation that is close to a real presentation.

[1960] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1961] The system of the present invention includes a means for a user to input the content of a presentation and automatically generate presentation materials using a generative AI model, a means for displaying the generated presentation materials, a means for analyzing a video of a practice presentation and providing feedback, a means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, a means for practicing the presentation in a scene set up using virtual reality, and a means for analyzing emotions by combining an emotion engine.

[1962] Document creation function

[1963] User Action:

[1964] The user enters the content of the presentation into a dedicated input form. For example, they can enter content such as "Marketing strategy for the next product." Once the input is complete, they click the "Submit" button.

[1965] The device:

[1966] The presentation content sent by the user is sent to the server. The input content is properly formatted.

[1967] The server:

[1968] The server analyzes the received presentation content and uses a generative AI model to automatically generate presentation materials, such as a title slide, marketing strategy details, market analysis, action plan, and conclusion slides.

[1969] The device:

[1970] The generated presentation materials are received in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[1971] Evaluation function

[1972] User Action:

[1973] Users record videos of their practice presentations and upload them to the system.

[1974] The device:

[1975] The recorded video file is sent to the server.

[1976] The server:

[1977] The received video is analyzed. Specifically, audio analysis, image analysis, and an emotion engine are used to analyze the user's emotions.

[1978] The server:

[1979] Based on the analysis results, the system evaluates the presentation's delivery style, tone of voice, speed, eye movements, posture, and even emotional aspects, and generates feedback, including specific suggestions for improvement such as "adjust your speaking speed" and "maintain a consistent gaze."

[1980] The device:

[1981] Feedback is received and displayed to the user, who can use it to improve their next practice.

[1982] Demonstration function

[1983] User Action:

[1984] Users upload their presentation materials and a sample of their voice.

[1985] The device:

[1986] These data are sent to the server.

[1987] The server:

[1988] Based on the presentation materials and audio samples, a generative AI model is used to generate a synthetic voice that resembles the user's voice. This synthetic voice is then used to generate a demonstration audio file using the presentation materials.

[1989] The device:

[1990] The system receives a demonstration audio file and plays it for the user, who can use it as a model to improve their presentation content and speaking style.

[1991] Virtual practice function for each scene

[1992] User Action:

[1993] Users select the VR practice mode and set the desired scene (e.g., a large hall, a conference room, etc.).

[1994] The device:

[1995] Send scene setting information to the server.

[1996] The server:

[1997] The specified virtual environment is generated based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[1998] The device:

[1999] The received VR environment data is transferred to the VR goggles.

[2000] User Action:

[2001] Users put on VR goggles and practice their presentations in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[2002] The role of the emotional engine

[2003] The server:

[2004] The server uses an emotion engine to collect emotional data from the user's facial expressions, tone of voice, and speaking style during presentation practice videos and real-time VR practice sessions, detecting emotions such as tension, joy, and impatience.

[2005] The server:

[2006] The emotion engine analyzes the emotional data detected and generates feedback based on that data, including specific advice such as "You seem nervous, so try slowing down your speech" or "Speak more emotionally."

[2007] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[2008] The processing flow will be explained below.

[2009] Document creation function

[2010] Processing Steps:

[2011] Step 1:

[2012] The user inputs the content of the presentation on their own terminal. The input content is a theme such as "New product market introduction plan."

[2013] Step 2:

[2014] After checking the input contents, the user clicks the "Send" button, which sends the input presentation contents from the terminal to the server.

[2015] Step 3:

[2016] The device formats the presentation content it receives and sends it to the server.

[2017] Step 4:

[2018] The server analyzes the received presentation content, and uses natural language processing technology to extract key points.

[2019] Step 5:

[2020] The server uses the generative AI model to automatically generate presentation materials, specifically, title slides, product features, market analysis, introduction schedule, and conclusion slides based on the selected template.

[2021] Step 6:

[2022] Save the generated presentation materials in PDF or PPT format and send them to your device.

[2023] Step 7:

[2024] The terminal displays the presentation materials received from the server to the user, who can then review the displayed materials and make corrections or additions as necessary.

[2025] Evaluation function

[2026] Processing Steps:

[2027] Step 1:

[2028] A user records a video of their practice presentation, which includes the user giving the presentation.

[2029] Step 2:

[2030] Users upload recorded videos to the system.

[2031] Step 3:

[2032] The device sends the uploaded video file to the server.

[2033] Step 4:

[2034] The server analyzes the received video. Specifically, it performs audio and image analysis, and then extracts emotional data using an emotion engine.

[2035] Step 5:

[2036] The server evaluates speaking style, tone of voice, speed, eye movements, posture, and emotions, and extracts evaluation points, such as "speaking too fast," "unfocused eye contact," and "appears nervous."

[2037] Step 6:

[2038] The server generates feedback based on the analysis results. The feedback includes suggestions for improving speaking style and posture, as well as emotional advice. For example, "You seem nervous, so try taking a deep breath" or "Try to lower your voice tone and speak calmly."

[2039] Step 7:

[2040] The server generates feedback and sends it to the device.

[2041] Step 8:

[2042] The device displays feedback to the user, who can then adjust their next practice based on that feedback.

[2043] Demonstration function

[2044] Processing Steps:

[2045] Step 1:

[2046] The user uploads the presentation materials they have created and a sample of their own voice (audio file) to the system.

[2047] Step 2:

[2048] The device sends the uploaded presentation materials and voice sample to the server.

[2049] Step 3:

[2050] Based on the presentation materials and voice samples received by the server, a generative AI model is used to generate synthetic voice that resembles the user's voice.

[2051] Step 4:

[2052] The server performs a demonstration using presentation materials with synthetic voice and generates a synthetic voice presentation.

[2053] Step 5:

[2054] The server transmits the generated demonstration audio file to the terminal.

[2055] Step 6:

[2056] The device plays the demonstration audio to the user, who can use it as a model to improve the content and speaking style of their presentation.

[2057] Virtual practice function for each scene

[2058] Processing Steps:

[2059] Step 1:

[2060] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.).

[2061] Step 2:

[2062] The device sends the scene setting information to the server.

[2063] Step 3:

[2064] The server generates a specified virtual environment based on the scene information received.

[2065] Step 4:

[2066] The VR environment data generated by the server is sent to the terminal and VR goggles.

[2067] Step 5:

[2068] The terminal transfers the received VR environment data to the VR goggles.

[2069] Step 6:

[2070] The user puts on VR goggles and practices their presentation in a virtual environment. The emotion engine analyzes the user's emotions during the presentation in real time and provides feedback based on that data.

[2071] In this way, the system of the present invention comprehensively improves the presentation skills of users through the rapid creation of presentation materials, the provision of detailed feedback, and advanced evaluation using emotional data.

[2072] Example 2

[2073] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2074] Traditional methods for creating, practicing, and evaluating presentation materials require a great deal of time and effort, making it difficult to improve the quality of presentations. In particular, in order to receive feedback on one's own performance, one must request evaluations from others, making it difficult to receive objective and continuous feedback. Furthermore, the content and structure of presentation materials must be created manually, making it difficult to generate materials quickly and effectively.

[2075] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2076] In this invention, the server includes: means for a user to input presentation content; means for using a generative AI model to automatically generate presentation materials based on the input content; means for displaying the generated presentation materials; means for analyzing a video of a practice presentation and providing feedback based on the results using audio analysis, image analysis, and the generative AI model; means for demonstrating a model presentation using a synthesized voice that resembles the user's voice; means for practicing the presentation in a scene set using virtual reality; and means for analyzing emotional data during the presentation using an emotion engine and providing feedback. This allows users to quickly and automatically create presentation materials and receive multifaceted feedback to improve the quality of their presentations.

[2077] "Presentation content" refers to the information and data that a user inputs to create presentation materials.

[2078] A "generative AI model" is an artificial intelligence algorithm that automatically creates presentation materials based on input content.

[2079] "Presentation Materials" means the content of a presentation in slide or other format, typically generated as a PDF or PPT file.

[2080] "Voice analysis" is a technology that analyzes recorded voice data and evaluates the tone, speed, quality of the voice, etc.

[2081] "Image analysis" is a technology that analyzes recorded video data and evaluates eye movements, facial expressions, posture, etc.

[2082] "Feedback" is an evaluation of the content and performance of a presentation and specific advice for improvement.

[2083] "Synthetic voice" is an artificial voice that resembles the user's voice and is generated based on a user's voice sample.

[2084] "Virtual reality" refers to a virtual digital environment generated using computer technology in which users can interact.

[2085] The "emotion engine" is a technology that analyzes a user's emotions from facial expressions, tone of voice, etc., and detects specific emotional states.

[2086] A "scene" refers to a specific environment or location within virtual reality where a presentation takes place.

[2087] MODE FOR CARRYING OUT THE INVENTION

[2088] The system of the present invention automatically generates presentation materials based on the content entered by the user in order to improve the quality of presentations, and provides multifunctional support for supporting presentation practice.

[2089] Document creation function

[2090] User operations

[2091] The user enters the content of the presentation into a dedicated input form, for example, a specific content such as "Marketing strategy for the next product," and clicks the submit button.

[2092] Device operation

[2093] The device formats the input presentation content appropriately and sends it to the server. The data sent is in text format and is encoded.

[2094] Server Operations

[2095] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. Specifically, slides are generated, including a title slide, details of the marketing strategy, market analysis, an action plan, and a conclusion. Various natural language processing technologies are used in this generative AI model.

[2096] Device operation

[2097] The generated presentation materials are sent to the terminal in PDF or PPT format and displayed to the user, who can then review the materials and make any necessary corrections.

[2098] Prompt Sentence Examples

[2099] "Please create a presentation about the marketing strategy for our next product."

[2100] Evaluation function

[2101] User operations

[2102] The user records a video of the presentation practice and uploads the video file to the system.

[2103] Device operation

[2104] The device sends the recorded video file to the server, where the file format and size are checked.

[2105] Server Operations

[2106] The server analyzes the received video using audio analysis, image analysis, and an emotion engine, and analyzes voice tone, speed, eye movement, posture, emotion, etc.

[2107] Server Operations

[2108] Feedback is generated based on the analysis results, including suggestions for improvement such as "adjust speaking speed" and "maintain consistent eye contact."

[2109] Device operation

[2110] The feedback is sent to the device and displayed to the user, who can use it to improve their next practice.

[2111] Prompt Sentence Examples

[2112] "Record yourself practicing your presentation and provide feedback."

[2113] Demonstration function

[2114] User operations

[2115] Users upload their own presentation materials and voice samples to the system.

[2116] Device operation

[2117] Your presentation and audio samples are sent to the server, where they are checked for correct file format and size.

[2118] Server Operations

[2119] The server uses a generative AI model to generate a synthetic voice that resembles the user's voice based on the uploaded voice sample, and then uses the generated synthetic voice to create a demonstration audio file of the presentation material.

[2120] Device operation

[2121] The demonstration audio file is sent to the terminal and played back for the user, who can use the audio file as a model to improve the content and speaking style of their presentation.

[2122] Prompt Sentence Examples

[2123] "Please convert your presentation materials into audio files and play them as a model."

[2124] Virtual practice function for each scene

[2125] User operations

[2126] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). Scenes can be selected from the system's catalog.

[2127] Device operation

[2128] Scene setting information is sent to the server, including the selected scene's ID and detailed parameters.

[2129] Server Operations

[2130] The server generates the specified virtual environment based on the received scene information, and the generated virtual environment data is sent to the device and VR goggles.

[2131] Device operation

[2132] The terminal transfers the received VR environment data to the VR goggles.

[2133] User operations

[2134] Users put on VR goggles and practice their presentation in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data.

[2135] Prompt Sentence Examples

[2136] "I want to practice my presentation in a virtual conference room environment."

[2137] By combining these functions, users can quickly create presentation materials and practice them with detailed feedback. Furthermore, by using an emotion engine, it is possible to analyze emotions such as tension and joy and provide more effective feedback.

[2138] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2139] Document creation function

[2140] Step 1:

[2141] User actions

[2142] The user enters the content of the presentation into a special input form and clicks the send button. The input is specific content such as "marketing strategy for the next product." The output is the content of the form being sent to the terminal as data.

[2143] Step 2:

[2144] Processing performed by the device

[2145] The device properly formats the presentation content received from the user and sends it to the server. The input is the text data of the presentation content sent by the user, and the output is sending the formatted data to the server. Specifically, the data is formatted, encoded, and sent to the server according to the communication protocol.

[2146] Step 3:

[2147] Processing performed by the server

[2148] The server analyzes the received presentation content and automatically generates presentation materials using a generative AI model. The input is the formatted presentation content, and the output is the generated presentation material. Specific operations include text analysis, determining the slide structure and layout, and generating automatically inserted images and graphs.

[2149] Step 4:

[2150] Processing performed by the device

[2151] The terminal receives the generated presentation materials in PDF or PPT format and displays them to the user. The input is the presentation material file sent from the server, and the output is to display that file on the user's device. Specific operations include downloading the file, displaying it in a viewer, and accepting edits.

[2152] Evaluation function

[2153] Step 1:

[2154] User actions

[2155] A user records a video of a practice presentation and uploads the video file to the system. The input is the recorded video file, and the output is the uploaded video data.

[2156] Step 2:

[2157] Processing performed by the device

[2158] The device sends the recorded video file to the server. The input is the video file uploaded by the user, and the output is to send this file to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[2159] Step 3:

[2160] Processing performed by the server

[2161] The server uses audio analysis, image analysis, and an emotion engine to analyze the received video. The input is the received video file, and the output is the analysis results and feedback. Specific operations include analyzing voice tone and speed, analyzing eye movements and posture, and detecting emotional states.

[2162] Step 4:

[2163] Processing performed by the server

[2164] Feedback is generated based on the analysis results. The input is the analysis results, and the output is the generated feedback. Specific operations include extracting areas for improvement based on the analysis data and generating specific feedback statements.

[2165] Step 5:

[2166] Processing performed by the device

[2167] The terminal receives the feedback data and displays it to the user. The input is the feedback data sent from the server, and the output is to show this data to the user. Specific operations include displaying feedback text, hints, and operation guides for the user's next practice.

[2168] Demonstration function

[2169] Step 1:

[2170] User actions

[2171] Users upload their created presentation materials and their own voice samples to the system. The input is the presentation material file and the voice sample file, and the output is the transfer of these data to the terminal.

[2172] Step 2:

[2173] Processing performed by the device

[2174] The terminal sends presentation materials and audio samples to the server. The input is the file uploaded by the user, and the output is to send these files to the server. Specific operations include checking the file format, compressing the file size, and communicating according to the transmission protocol.

[2175] Step 3:

[2176] Processing performed by the server

[2177] The server uses a generative AI model to generate synthetic speech that resembles the user's voice based on the uploaded voice sample. This synthetic speech is then used to create a demonstration audio file of the presentation materials. The input is the voice sample and the presentation materials, and the output is the synthesized demonstration audio file. Specific operations include speech synthesis, audio file generation, and reading of the presentation materials.

[2178] Step 4:

[2179] Processing performed by the device

[2180] Receives a demonstration audio file and plays it for the user. The input is the audio file sent from the server, and the output is the playback of the audio file. Specific operations include downloading the file, playing it in an audio player, and accepting user operations.

[2181] Virtual practice function for each scene

[2182] Step 1:

[2183] User actions

[2184] The user selects the VR practice mode and sets the desired scene (e.g., a large hall, a conference room, etc.). The input is the scene setting information, and the output is the set information being sent to the device.

[2185] Step 2:

[2186] Processing performed by the device

[2187] Sends scene setting information to the server. The input is the scene information set by the user, and the output is to send this information to the server. Specific operations include formatting the setting information and communicating it according to the transmission protocol.

[2188] Step 3:

[2189] Processing performed by the server

[2190] The server generates a virtual environment based on the received scene information. The input is the received scene information, and the output is the generated virtual environment data. Specific operations include 3D modeling of the virtual environment and scenario setting.

[2191] Step 4:

[2192] Processing performed by the device

[2193] The received VR environment data is transmitted to the VR goggles. The input is the virtual environment data sent from the server, and the output is the data transmitted to the VR goggles. The specific operations include decoding the data and transmitting it to the device.

[2194] Step 5:

[2195] User actions

[2196] Users put on VR goggles and practice their presentations in a virtual environment. During the presentation, the emotion engine analyzes emotions in real time and provides feedback based on that data. The input is the actual presentation's movements and emotional data, and the output is real-time feedback.

[2197] The above are the processing steps in the system of the present invention, which allows users to achieve comprehensive improvement in their presentation skills.

[2198] (Application example 2)

[2199] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2200] Conventional presentation practice support systems incorporate functions for automatically generating presentation materials and providing feedback on practice, but they are not adequately suited to real-world work environments, particularly in factory environments. Furthermore, because they do not provide real-time feedback based on emotional data, it is difficult to effectively alleviate issues such as tension and impatience. This results in problems such as reduced user work efficiency and reduced teaching effectiveness.

[2201] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2202] In this invention, the server includes means for a user to input presentation content, means for automatically generating presentation materials based on the input content, means for displaying the generated presentation materials, means for analyzing a video of a practice presentation and providing feedback based on the results, means for demonstrating a model presentation using a synthesized voice that resembles the user's voice, means for practicing the presentation in a scene set using virtual reality, means for analyzing a recorded video of the practice presentation, collecting emotional data and providing feedback, and means for providing work guidance in a factory using virtual reality and robots. This allows users to practice in a simulation that resembles an actual work environment, and by receiving feedback based on the emotional data, the quality of their presentations can be improved.

[2203] "User" refers to a person who uses the presentation system to create presentation materials, practice them, and receive feedback.

[2204] "Presentation content" refers to the information and outline that a user enters when creating presentation materials.

[2205] "Auto-generation" refers to the process of automatically generating presentation materials using machine learning models and algorithms based on input information and prompts.

[2206] "Presentation materials" refers to materials such as slides and documents used to effectively communicate the content of a presentation.

[2207] "Display means" refers to a device or method for visually presenting the generated presentation materials to the user.

[2208] "Feedback" means information or comments that provide improvements or evaluations based on the results of your practice presentation.

[2209] "Synthetic voice" refers to a voice that resembles a human voice and is generated using voice synthesis technology.

[2210] "Virtual reality" means a virtual environment created using computer graphics and other technologies, in which users can have a realistic experience.

[2211] "Emotional data" refers to information about the user's emotional state analyzed from facial expressions, voice, etc.

[2212] "Work instruction" means the process of providing instruction and training regarding work tasks and procedures within a factory.

[2213] A "robot" refers to an automated machine that performs programmed actions and provides information and work instructions to users.

[2214] The system of the present invention allows users to input presentation content, automatically generate presentation materials using a generative AI model, display and evaluate the materials, and practice the presentation in a virtual reality environment. To effectively implement the system of the present invention, the following hardware and software are used:

[2215] Hardware and software used

[2216] 1. Hardware

[2217] Server: High-performance data processing server

[2218] Device: A computer or tablet that a user operates.

[2219] VR system: Virtual reality goggles such as Oculus Quest 2

[2220] Robot: A multi-functional robot that provides guidance and feedback within the fa...

Claims

1. a means for a user to input the content of a presentation; means for automatically generating presentation materials based on input content; a means for displaying the generated presentation material; A method to analyze videos of presentation practice and provide feedback based on the results, A means to demonstrate a model presentation using a synthesized voice that resembles the user's voice, A way to practice presentations in virtual reality settings, A system including:

2. 2. The system according to claim 1, wherein the means for analyzing the video of the presentation practice extracts evaluation points by audio analysis and image analysis, and provides points for improvement and correction.

3. 10. The system of claim 1, wherein the virtual reality scene simulates a particular presentation environment, such as a large auditorium or conference room.

Images