system

Abstract

We provide a system that centrally manages multiple tasks and enables their efficient execution. [Solution] A system including means for users to input schedule information from each terminal and save it in a central database; means for users to input meeting minutes during a meeting and send them to the server after the meeting ends; means for users to create plans and reports, save them after completion, and notify relevant parties; means for users to set market research questions, distribute questionnaires, and collect and analyze response data; means for managing employee training curricula, distributing training modules, and reporting on progress; means for users to formulate video content and shooting plans, edit them, and save the final version to the server; means for collecting system operation logs, detecting and reporting problems; and means for inputting and recording inquiry details to call center operators.

Description

Technical Field

[0004] , ,

[0005] , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern business environment, efficient management and execution of various operations are required. However, there is no means for unified management of a wide range of operations such as schedule management, minutes of meetings creation, plan / report creation, market research, employee education, promotion video creation, system test / monitoring, and call center response. As a result, these operations are carried out individually, leading to reduced efficiency and inability to maintain information consistency. Consequently, there are problems such as duplication of operations and omission of information, which have an adverse impact on the productivity of the enterprise.

Means for Solving the Problems

[0005] This invention relates to a system for centrally managing and efficiently executing multiple tasks. Specifically, it provides a system that includes the following means.

[0006] A means for users to input schedule information from each device and save it to a central database.

[0007] A method for users to enter meeting minutes during a meeting and send them to the server after the meeting ends.

[0008] A means for users to create plans and reports, save them after completion, and notify relevant parties.

[0009] A means for users to set market research questions, distribute questionnaires, and collect and analyze response data.

[0010] A means of managing employee training curricula, distributing training modules, and reporting on progress.

[0011] A means for users to formulate video content and shooting plans, edit them, and save the final version to a server.

[0012] A means of collecting system operation logs, detecting problems, and reporting them.

[0013] A means for call center operators to input and record the content of inquiries.

[0014] This will integrate various business processes, improve coordination between them, and ensure corporate productivity and information consistency.

[0015] A "user" refers to an entity that uses this system to manage and execute various tasks.

[0016] "Terminal" refers to devices such as computers, smartphones, and tablets that users operate.

[0017] "Server" refers to a central system that receives information from terminals and stores and processes data.

[0018] "Schedule information" refers to data on plans and schedules input by users.

[0019] "Central database" refers to a data management system for centrally storing schedule information and other data.

[0020] "Minutes of meeting" refers to a document recording the discussion content and decisions during a meeting.

[0021] "Plan" refers to a document describing the details of a new project or idea.

[0022] "Report" refers to a document summarizing the results of specific operations or investigations.

[0023] "Market research" refers to activities for collecting and analyzing information on specific markets and customers.

[0024] "Questionnaire" refers to a survey method for collecting responses to specific questions.

[0025] "Response data" refers to the response content of questionnaires collected from survey respondents.

[0026] "Employee education" refers to educational and training activities for improving employees' skills and knowledge.

[0027] "Curriculum" refers to the learning content and schedule planned for education.

[0028] "Educational module" refers to an individual learning unit provided as part of a curriculum.

[0029] "Promotion video" refers to video content produced for the purpose of promoting products or services.

[0030] "Testing" refers to experiments conducted to verify the functionality and performance of a system or software.

[0031] "Monitoring" refers to the activity of continuously monitoring the operating status and performance of a system.

[0032] A "call center" refers to a service that handles customer inquiries and support requests.

[0033] An "operator" refers to a person who handles customer inquiries at a call center.

[0034] "Inquiry content" refers to the specific questions or problems that customers bring to the call center for consultation. [Brief explanation of the drawing]

[0035] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0036] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0037] First, let's explain the terminology used in the following explanation.

[0038] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0039] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0040] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0041] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0042] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0043] [First Embodiment]

[0044] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0045] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0046] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0047] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0048] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0049] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0050] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0051] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0052] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0053] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0054] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0055] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0056] This invention relates to a system for centrally managing and efficiently executing various tasks. This system enables the efficient operation of a wide range of tasks through the collaboration of users, terminals, and servers.

[0057] Daily schedule management

[0058] 1. Schedule entry:

[0059] Users enter their daily schedules and tasks from each device.

[0060] The terminal temporarily saves the entered schedule and sends it to the server.

[0061] 2. Schedule saving and synchronization:

[0062] The server stores the received schedule information in a central database.

[0063] The server synchronizes the latest schedule information to each terminal after saving it.

[0064] Meeting minutes

[0065] 1. Entering meeting minutes:

[0066] The user enters meeting minutes into their terminal during the meeting.

[0067] The terminal temporarily saves meeting minutes in real time.

[0068] 2. Sending and saving meeting minutes:

[0069] The user clicks the save button after the meeting ends.

[0070] The terminal sends the meeting minutes to the server.

[0071] The server saves the received meeting minutes to a central database.

[0072] Planning and report writing

[0073] 1. Planning and input:

[0074] Users create drafts of plans and reports using their devices.

[0075] The terminal provides users with necessary templates and historical documents.

[0076] 2. Saving and Notifications:

[0077] The terminal sends completed plans and reports to the server.

[0078] The server stores them in a central database and sends notifications to the relevant parties.

[0079] Market research

[0080] 1. Creating and preparing the questionnaire:

[0081] The user sets the market research questions on their device.

[0082] The terminal sends the configured question to the server.

[0083] 2. Distribution of questionnaires:

[0084] The server generates a survey link and sends it to each recipient based on the distribution list.

[0085] 3. Response collection and analysis:

[0086] Participants will answer the questionnaire via the link they receive.

[0087] The device sends the response data to the server.

[0088] The server analyzes the collected response data, generates a visual report, and provides it to the user.

[0089] Employee training

[0090] 1. Curriculum management and delivery:

[0091] The server manages the employee training curriculum and distributes training modules to each terminal.

[0092] 2. Report on educational progress:

[0093] The terminal reports the progress of the educational module to the server in real time.

[0094] Users enter feedback after each session is completed and send it to the server.

[0095] Creating a promotional video

[0096] 1. Planning and developing video content:

[0097] The user uses the device to plan the video content and shooting schedule.

[0098] 2. Filming and editing:

[0099] The terminal uses video editing software to edit the footage and sends the final version to the server.

[0100] The server stores the completed video and generates the necessary links to provide to the user.

[0101] System testing and monitoring

[0102] 1. Run the test:

[0103] The terminal executes the test script and sends the results to the server.

[0104] 2. Log collection and monitoring:

[0105] The server collects operation logs from each terminal and monitors for anomalies.

[0106] The server will report any problems detected to the user.

[0107] Call center initial support

[0108] 1. Inquiry handling:

[0109] The operator enters the customer's inquiry into the terminal.

[0110] 2. Recording and Analysis:

[0111] The server stores the query content in a central database and performs analysis.

[0112] Specific example:

[0113] For example, when a user uses the "market research" function, the process proceeds as follows:

[0114] 1. The user sets the survey questions on their device and submits them.

[0115] 2. The server receives the questions and generates a survey link.

[0116] 3. The device sends the link to the person being surveyed.

[0117] 4. Participants will answer the questionnaire using the provided link.

[0118] 5. The device sends the response data to the server.

[0119] 6. The server aggregates the data, generates a report including the analysis results, and provides it to the user.

[0120] This allows the system to help users conduct market research effectively.

[0121] The following describes the processing flow.

[0122] Market research processing steps

[0123] Creating and preparing the questionnaire for distribution

[0124] Step 1:

[0125] The user enters market research survey questions from their device.

[0126] The terminal temporarily stores the entered questions in local storage and checks the integrity of the input.

[0127] Step 2:

[0128] The terminal sends the questions that have completed the integrity check to the server.

[0129] The server reviews the received questions and saves them to the central database.

[0130] Distribution of questionnaires

[0131] Step 3:

[0132] The server generates a link to the survey and emails the link to each survey participant based on the distribution list.

[0133] Step 4:

[0134] The terminal reports the transmission status to the server in real time.

[0135] The server confirms that the survey link has been successfully sent and updates the log.

[0136] Collection of responses

[0137] Step 5:

[0138] Participants click the received link to access the survey page.

[0139] The device displays the survey response screen, and the survey participants answer the questions.

[0140] Step 6:

[0141] The device temporarily stores the survey participants' responses in local storage and sends the data to the server once all responses have been completed.

[0142] Step 7:

[0143] The server saves the received response data to a central database and returns a confirmation message to the terminal acknowledging the successful saving.

[0144] Data aggregation and analysis

[0145] Step 8:

[0146] The server aggregates the stored response data and performs statistical analysis.

[0147] The server converts the analysis results into graphs and charts, generating visual reports.

[0148] Step 9:

[0149] The server sends the generated report to the terminal.

[0150] Users can view reports from their devices and provide feedback as needed.

[0151] Record of feedback

[0152] Step 10:

[0153] The device sends the user's input to the server.

[0154] The server saves the received feedback to a database and records it as an area for improvement in future market research.

[0155] The above processing steps enable the system to efficiently carry out the entire market research process, from data collection and analysis to reporting, in a consistent manner.

[0156] (Example 1)

[0157] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0158] Currently, there is a need for a centralized management system to efficiently handle multiple functions in business management and operations, such as schedule management, meeting minute creation, planning and report creation, market research, employee training, promotional video production, system testing and monitoring, and call center inquiry handling. When these tasks are managed individually, data synchronization and sharing become cumbersome, leading to decreased efficiency. Furthermore, there is a growing need for a system that reduces user effort while enabling accurate and rapid data processing.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0160] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for terminals to temporarily store schedules and send them to the server, and means for the server to store schedule information in a central database and synchronize it with each terminal. This enables centralized management of schedule information and real-time data synchronization. It also includes means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to create plans and reports, save them after completion and notify relevant parties, and means for users to set market research questions, distribute questionnaires, and collect and analyze response data. The server also includes means for generating questionnaire links and sending them to each target based on a distribution list, means for managing employee training curricula, distributing training modules and reporting progress, means for users to formulate and edit video content and shooting plans and save the final version to the server, means for the server to collect system operation logs, detect and report problems, and means for call center operators to input and record inquiry details. This enables efficient and accurate centralized management of multiple tasks, significantly reducing user effort and improving overall operational efficiency.

[0161] A "user" refers to an entity that uses the system to perform various tasks.

[0162] A "terminal" refers to a device operated by a user, such as a computer or smartphone used for tasks like entering schedules, creating meeting minutes, or designing surveys.

[0163] A "server" refers to a central computer system used to store data and manage communication and data synchronization between terminals.

[0164] A "central database" refers to a database built on a server that centrally manages and stores all data generated within the system.

[0165] "Schedule information" refers to data about appointments and tasks entered by the user.

[0166] "Meeting minutes" refers to a document that users record during a meeting, including details of the meeting, discussions, and decisions made.

[0167] "Plans" and "reports" refer to plans and reports created by users, which are documents containing information about the business or project.

[0168] "Market research" refers to the process of collecting market reactions and opinions through surveys set by users.

[0169] A "survey link" is a URL generated by the server and sent to the survey participant, used to access the survey form.

[0170] An "employee training curriculum" refers to an educational program aimed at improving employees' skills and acquiring knowledge.

[0171] An "education module" is a part of the employee training curriculum and refers to the specific content of each lesson or session.

[0172] "Video content" refers to the scenario and structure that the user develops when creating a promotional video.

[0173] A "shooting plan" refers to a specific plan for shooting a video, including the shooting schedule, locations, and necessary equipment.

[0174] "Operation log" refers to data that records the operating status of a system or terminal.

[0175] A "call center operator" refers to a person who handles customer inquiries at a call center.

[0176] This invention relates to a system for centrally managing and efficiently executing various tasks. Through collaboration between users, terminals, and servers, it enables the efficient operation of a wide range of tasks.

[0177] Hardware and software to be used

[0178] hardware

[0179] Devices: Laptops, smartphones (iOS / ANDROID®), tablets (iPad®), desktop PCs

[0180] Server: Cloud server (AWS®, Azure®, Google® Cloud)

[0181] software

[0182] Database management systems (PostgreSQL, MySQL (registered trademark))

[0183] Text editor

[0184] Video editing software

[0185] Communication method (Firebase Cloud Messaging, HTTPS)

[0186] Email sending system (SendGrid)

[0187] Analysis tools (Python Pandas, Matplotlib)

[0188] The specific embodiments of each function of this system are described below.

[0189] Schedule management

[0190] Input and data transmission

[0191] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[0192] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[0193] Data storage and synchronization

[0194] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[0195] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[0196] Market research

[0197] Questionnaire creation and distribution

[0198] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[0199] The terminal POSTs the question data to the server in JSON format.

[0200] Generate and send survey link

[0201] The server uses a URL shortening service to generate the survey link.

[0202] The server will send this link to the recipients using the email sending system (SendGrid).

[0203] Response collection and analysis

[0204] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[0205] The device temporarily stores the response data and sends it to the server in JSON format.

[0206] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[0207] Example of a prompt

[0208] Schedule management:

[0209] "Please add a 10:00 meeting to this week's schedule and save it to the server."

[0210] Market research:

[0211] "Please create a questionnaire about the new product and send it to the target audience."

[0212] The various functions of this system allow users to efficiently centralize and manage multiple tasks, significantly reducing effort and improving overall operational efficiency.

[0213] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0214] Daily schedule management

[0215] Step 1:

[0216] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[0217] Input: Date, time, meeting content (e.g., 10:00, meeting with a client)

[0218] Action: The user creates a new schedule in the calendar app and clicks the save button.

[0219] Output: Schedule data stored in the local cache

[0220] Step 2:

[0221] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[0222] Input: Schedule data (e.g., 10:00, Meeting with a client)

[0223] Operation: Save to local cache and convert to JSON format.

[0224] Output: POST request to the server

[0225] Step 3:

[0226] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[0227] Input: Schedule data in JSON format

[0228] Operation: Data analysis and insertion into a PostgreSQL database.

[0229] Output: Schedule data stored in the database

[0230] Step 4:

[0231] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[0232] Input: Schedule data stored in the database

[0233] Operation: Triggering push notifications using Firebase Cloud Messaging

[0234] Output: Latest schedule notification received on each device

[0235] Market research

[0236] Step 1:

[0237] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[0238] Input: Question item (e.g., What do you think of the product?)

[0239] Operation: Enter and save a new question on the web interface.

[0240] Output: Question data stored in the local cache

[0241] Step 2:

[0242] The terminal POSTs the question data to the server in JSON format.

[0243] Input: Question data (e.g., What do you think of the product?)

[0244] Operation: Save to local cache and convert to JSON format.

[0245] Output: POST request to the server

[0246] Step 3:

[0247] The server uses a URL shortening service to generate the survey link.

[0248] Input: Question data

[0249] Operation: Generates survey links using a URL shortening service.

[0250] Output: Generated survey link

[0251] Step 4:

[0252] The server will send this link to the recipients using the email sending system (SendGrid).

[0253] Input: Generated survey link

[0254] Operation: Send emails based on distribution lists using SendGrid.

[0255] Output: Email and survey link sent to participants

[0256] Step 5:

[0257] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[0258] Input: Response data entered by survey participants.

[0259] How it works: Fill out the survey form in your web browser and submit your response.

[0260] Output: Sending response data to the server

[0261] Step 6:

[0262] The device temporarily stores the response data and sends it to the server in JSON format.

[0263] Input: Response data

[0264] Operation: Save to local cache and convert to JSON format.

[0265] Output: POST request to the server

[0266] Step 7:

[0267] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[0268] Input: Response data in JSON format

[0269] Function: Saves to database, runs analytical algorithms, generates visual reports (Pandas, Matplotlib)

[0270] Output: Visual reports displayed on the user's dashboard.

[0271] Example of a prompt

[0272] Schedule Management: "Add a 10:00 meeting to this week's schedule and save it to the server."

[0273] Market research: "Create a questionnaire about the new product and send it to the target audience."

[0274] This program's processing steps allow users to efficiently manage schedules and conduct market research, enabling them to manage various tasks in an integrated manner.

[0275] (Application Example 1)

[0276] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0277] Traditional business management systems have struggled to efficiently integrate scheduling, real-time tracking of operations, meeting minute creation, employee training, and system testing and monitoring within logistics centers. In particular, the inability to track the movement of goods and the progress of picking lists within the logistics center in real time led to decreased operational efficiency and increased errors. A new system is needed to address these challenges.

[0278] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0279] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to set market research questions, distribute questionnaires, and collect and analyze response data, and means for tracking the movement of goods within the logistics center and the progress of picking lists in real time. This enables efficient operation of the entire logistics center and real-time management of various operations.

[0280] A "user" is an entity that uses a system to perform various tasks.

[0281] A "terminal" is a device used by a user to input information or utilize the functions of a system.

[0282] A "central database" is a storage device for centrally storing information of the entire system.

[0283] A "server" is a computer system for receiving and processing data transmitted from each terminal.

[0284] "Schedule information" is information on daily schedules and tasks input by a user.

[0285] "Meeting minutes" are the contents recorded by a user during a meeting and are documents saved after the meeting.

[0286] A "plan" is a new project or plan created by a user.

[0287] A "report" is a document summarizing the progress and results of a plan.

[0288] "Market research" is an activity of collecting and analyzing data on a specific market.

[0289] A "questionnaire" is a survey in the form of questions conducted as part of market research.

[0290] "Data collection" is the act of collecting information transmitted from each terminal.

[0291] "Data analysis" is the process of verifying the collected information and summarizing it as a visual report.

[0292] "Employee education curriculum" is a learning program aimed at improving employees' skills.

[0293] An "education module" refers to software and teaching materials used to implement employee training curricula.

[0294] "Feedback" refers to the evaluations and opinions that users enter after completing an educational module.

[0295] A "promotional video" is video content produced for the purpose of advertising or public relations.

[0296] "Real-time tracking" is a function that allows you to instantly grasp the status of ongoing operations and logistics.

[0297] A "picking list" is a list that displays the items needed for a specific order or task.

[0298] This invention is a system for efficiently managing and operating operations at a logistics center. The specific program and its processing details for realizing this system are described below.

[0299] System Configuration

[0300] This system consists primarily of users, production terminals, an information processing server, and a central database. The terminals provide an interface for users to input information and send it to the server. The server stores the received information in the central database and synchronizes it with other terminals as needed.

[0301] Schedule management

[0302] Users input their daily tasks and schedules within the logistics center using terminals. This schedule information is temporarily stored on the terminals and then sent to the server. The server stores the received schedule information in a central database and synchronizes the latest schedule information with each terminal.

[0303] Real-time tracking

[0304] To track the movement of items within the logistics center and the progress of picking lists in real time, data is input by terminals. The terminals send this information to the server in real time, and the server saves it in the central database to improve the efficiency of item management.

[0305] Meeting Minutes Creation

[0306] During the meeting, the user inputs the meeting minutes into the terminal. After the meeting, the user clicks the save button, and the meeting minutes are sent from the terminal to the server. The server saves the received meeting minutes in the central database and sends notifications to the relevant personnel.

[0307] Employee Training

[0308] The server manages the employee training curriculum and distributes training modules to each terminal. Reports on training progress are sent from the terminals to the server in real time. The user inputs feedback after each session and sends it to the server.

[0309] System Testing and Monitoring

[0310] The terminal collects the system operation logs and sends them to the server. The server saves the operation logs of each terminal in the central database and reports to the user if an anomaly is detected.

[0311] Specific Example

[0312] For example, when adding a new inventory check task, the user inputs as follows from the schedule input screen.

[0313] Example of Schedule Input Prompt

[0314] Please add a new task.

[0315] User: employee1

[0316] Task name: Inventory check

[0317] Date: 2023-10-10

[0318] Time: 14:00

[0319] In this way, the system enables efficient operation of the entire logistics center and real-time management of various tasks. Furthermore, it can streamline other incidental tasks such as schedule management and meeting minute creation.

[0320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0321] Step 1:

[0322] The user opens the schedule entry screen using their device and enters their daily tasks and appointments. The entered data includes the username, task name, date, and time. The device temporarily saves the entered data and then sends it to the server.

[0323] Step 2:

[0324] The server receives schedule information sent from the terminals. The data includes the username, task name, date, and time, and this data is stored in a central database. After the schedule information is stored, the server sends a request to each terminal to synchronize with the latest schedule information.

[0325] Step 3:

[0326] Users use terminals to input information about the movement of items within the logistics center and picking lists in real time. The entered data is immediately sent from the terminal to the server.

[0327] Step 4:

[0328] The server receives real-time tracking data sent from the terminal and stores it in a central database. Based on the received data, the server monitors the movement of items and the progress of picking lists in real time.

[0329] Step 5:

[0330] During a meeting, the user enters meeting minutes using their device. After the meeting ends, the user clicks the save button on their device. The device temporarily saves the meeting minutes and sends them to the server.

[0331] Step 6:

[0332] The server receives meeting minutes data sent from terminals and stores it in a central database. This data includes the meeting title, agenda, participants, and minutes content. The server notifies relevant parties that the minutes have been saved.

[0333] Step 7:

[0334] Users access the employee training curriculum from their devices and utilize the training modules. After completing each session, users provide feedback. The device then sends the feedback and progress report to the server.

[0335] Step 8:

[0336] The server receives progress data and feedback on educational modules sent from terminals and stores them in a central database. The server manages educational progress and feedback in real time.

[0337] Step 9:

[0338] The terminal collects system operation status as logs and periodically sends them to the server. The log data includes operational anomalies, error messages, and performance metrics.

[0339] Step 10:

[0340] The server receives operation logs sent from the terminal and stores them in a central database. The server analyzes the log data and issues a warning to the user if an anomaly is detected.

[0341] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0342] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[0343] Emotional engine integration

[0344] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[0345] Daily schedule management

[0346] 1. Schedule entry:

[0347] Users enter their daily schedules and tasks from each device.

[0348] The terminal temporarily stores the entered schedule information and sends it to the server.

[0349] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data, which is then sent to the server.

[0350] 2. Schedule saving and synchronization:

[0351] The server stores schedule information and sentiment data in a central database.

[0352] The server synchronizes the latest schedule information and emotional feedback to each terminal.

[0353] Meeting minutes

[0354] 1. Entering meeting minutes:

[0355] The user enters meeting minutes into their terminal during the meeting.

[0356] The terminal temporarily saves meeting minutes in real time.

[0357] The emotion engine recognizes the user's emotions while they are typing and detects tension and stress.

[0358] 2. Sending and saving meeting minutes:

[0359] The user clicks the save button after the meeting ends.

[0360] The terminal sends meeting minutes and sentiment data to the server.

[0361] The server stores the received meeting minutes and sentiment data in a central database.

[0362] Planning and report writing

[0363] 1. Planning and input:

[0364] Users create drafts of plans and reports using their devices.

[0365] The terminal provides users with necessary templates and historical documents.

[0366] The emotion engine analyzes whether the user is experiencing stress or satisfaction.

[0367] 2. Saving and Notifications:

[0368] The terminal sends completed plans and reports to the server.

[0369] The server stores them in a central database and sends notifications to the relevant parties.

[0370] The emotion engine provides feedback that responds to the user's emotions.

[0371] Market research

[0372] 1. Creating and preparing the questionnaire:

[0373] The user sets the market research questions on their device.

[0374] The terminal sends the configured question to the server.

[0375] 2. Distribution of questionnaires:

[0376] The server generates the survey link and sends the link to each survey participant based on the distribution list.

[0377] The emotion engine analyzes the user's reaction when a link is sent.

[0378] 3. Response collection and analysis:

[0379] Participants click the received link to access the survey page.

[0380] The device displays the survey response screen, and the survey participants answer the questions.

[0381] The device sends the response data to the server.

[0382] The server analyzes the collected response data and generates a visual report.

[0383] The emotion engine analyzes emotional tendencies based on aggregated data and reflects them in the results.

[0384] Employee training

[0385] 1. Curriculum management and delivery:

[0386] The server manages the employee training curriculum and distributes training modules to each terminal.

[0387] The emotion engine adjusts the curriculum based on employee emotional data.

[0388] 2. Report on educational progress:

[0389] The terminal reports the progress of the educational module to the server in real time.

[0390] Users enter feedback after each session is completed and send it to the server.

[0391] The server receives feedback and sentiment data and analyzes the educational effect.

[0392] Creating a promotional video

[0393] 1. Planning and developing video content:

[0394] The user uses the device to plan the video content and shooting schedule.

[0395] The emotion engine analyzes the user's emotions and reflects them in the video content.

[0396] 2. Filming and editing:

[0397] The terminal uses video editing software to edit the footage and sends the final version to the server.

[0398] The server stores the completed video and generates the necessary links to provide to the user.

[0399] System testing and monitoring

[0400] 1. Run the test:

[0401] The terminal executes the test script and sends the results to the server.

[0402] 2. Log collection and monitoring:

[0403] The server collects operation logs from each terminal and monitors for anomalies.

[0404] The emotion engine evaluates the user's emotions while using the system and optimizes the user experience.

[0405] Call center initial support

[0406] 1. Inquiry handling:

[0407] The operator enters the customer's inquiry into the terminal.

[0408] The emotion engine analyzes the operator's emotions during interactions and provides advice for stress reduction.

[0409] 2. Recording and Analysis:

[0410] The server stores the query content in a central database and performs analysis.

[0411] The emotion engine combines collected data with emotional data to evaluate the quality of customer service.

[0412] Specific example:

[0413] For example, when a user uses the "market research" function, the process proceeds as follows:

[0414] 1. The user inputs market research questions from their device. The emotion engine analyzes the user's facial expressions and voice as they input the information.

[0415] 2. The terminal sends the entered questions to the server. The server stores the questions and sentiment data in a central database.

[0416] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[0417] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[0418] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[0419] This allows the system to help users conduct market research effectively and to provide higher quality services by utilizing sentiment data.

[0420] The following describes the processing flow.

[0421] Processing steps in market research

[0422] Creating and preparing the questionnaire for distribution

[0423] Step 1:

[0424] The user enters market research questions from their terminal.

[0425] The device temporarily stores the entered questions and sends them to the emotion engine.

[0426] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data.

[0427] The emotion engine sends emotional data back to the device.

[0428] Step 2:

[0429] The device sends the questionnaire items and sentiment data to the server.

[0430] The server stores the questionnaire items and sentiment data in a central database.

[0431] Distribution of questionnaires

[0432] Step 3:

[0433] The server generates a survey link and sends the link via email to each survey participant based on the distribution list.

[0434] Step 4:

[0435] The terminal reports the transmission status to the server in real time.

[0436] The server logs the successful submission of the survey link.

[0437] The emotion engine re-analyzes the user's response during this process and updates the emotion data.

[0438] Collection of responses

[0439] Step 5:

[0440] Participants in the survey will access the questionnaire page by clicking the link they receive.

[0441] The device displays the survey response screen, and the survey participants answer the questions.

[0442] Step 6:

[0443] The device temporarily stores the survey participants' responses and sends the data to the server once all responses have been completed.

[0444] Step 7:

[0445] The server stores the response data in a central database.

[0446] The emotion engine analyzes the emotional data of the survey participants on the server and stores it together with the data in the database.

[0447] The server sends a confirmation message to the terminal acknowledging that the save was successful.

[0448] Data aggregation and analysis

[0449] Step 8:

[0450] The server aggregates the stored response data and performs statistical analysis.

[0451] The emotion engine works with the server to analyze emotional tendencies based on aggregated data and reflect them in the results.

[0452] Step 9:

[0453] The server converts the analysis results into graphs and charts, generating visual reports.

[0454] The server sends the generated report and sentiment analysis results to the terminal.

[0455] Step 10:

[0456] Users can view reports from their devices and provide feedback as needed.

[0457] The device sends user feedback to the emotion engine.

[0458] The emotion engine analyzes feedback and emotional data to derive areas for improvement for the next market research.

[0459] The emotion engine sends the analysis results to the server and stores them in the database.

[0460] This allows the system to efficiently manage the entire market research process and leverage user sentiment data to provide high-quality feedback and analytical results.

[0461] (Example 2)

[0462] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0463] Traditional business management systems focus on individual tasks and schedule management, but lack mechanisms to provide feedback that takes into account the user's emotional state. Furthermore, few systems utilize emotional data to improve work efficiency and quality. As a result, there is no effective method for managing user stress and satisfaction, making it a challenge to improve work quality and employee mental health.

[0464] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database along with emotional data; means for users to input meeting minutes during a meeting, perform emotional analysis, and send them to the server after the meeting ends; and means for users to create plans and reports, receive emotional feedback, and then save them and notify relevant parties after completion. This enables the analysis of users' emotional data in real time and provides optimal feedback based on that analysis, thereby improving work efficiency and the mental health of employees.

[0465] "Schedule information" refers to appointments and action plans that users enter to manage their daily work and tasks.

[0466] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions, voice, behavioral patterns, and other factors.

[0467] A "central database" is a database used to centrally manage and store data for the entire system.

[0468] Meeting minutes are documents that record the content discussed and decisions made during a meeting.

[0469] "Emotion analysis" is the process of recognizing a user's emotional state based on their input and actions.

[0470] "Planning" refers to a concrete plan or concept for achieving a specific objective.

[0471] A "report" is a document that summarizes the progress and results of work or a project.

[0472] A "survey" is a method of collecting responses to questions set up for a specific research purpose.

[0473] "Employee training" refers to the provision of training and curricula designed to improve the skills and knowledge of employees and staff.

[0474] An "educational module" is a unit of learning content designed for a specific educational purpose.

[0475] "Video content" refers to the content of videos filmed for purposes such as promotion, explanation, or education.

[0476] A "shooting plan" is a plan that is formulated in advance to ensure efficient video shooting.

[0477] An "operation log" refers to a record of operations and events performed by a system or terminal.

[0478] A "call center" is a department or organization that functions as a point of contact for handling customer inquiries.

[0479] "Inquiry details" refers to the specific content of questions and consultations received by customers at the call center.

[0480] "Stress management" refers to methods for evaluating the stress that users and employees experience at work and appropriately mitigating it.

[0481] A "visual report" is a report that presents data in a visually easy-to-understand format, such as graphs and charts.

[0482] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[0483] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[0484] Hardware and software to be used

[0485] Emotion engine: Uses Microsoft® Azure Cognitive Services or Google Cloud Emotion AI.

[0486] Servers: We will use AWS EC2 servers and Amazon RDS as the database service.

[0487] Devices: iOS / Android devices are accepted.

[0488] Video editing software: Adobe Premiere Pro will be used.

[0489] Log collection software: Use the ELK stack (ElasticSearch®, Kibana, Logstash).

[0490] Emotional engine integration

[0491] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. This recognized emotion information is sent to the server and used in various business processes.

[0492] Daily schedule management

[0493] Users input their daily schedules and tasks from their respective devices, and their facial expressions and voices are analyzed by an emotion engine. The devices temporarily store the entered schedule information and send it to the server along with the emotion data. The server stores the schedule information and emotion data in a central database and synchronizes the latest schedule information and emotion feedback to each device.

[0494] Meeting minutes

[0495] During a meeting, the user enters meeting minutes into a terminal. The emotion engine recognizes the user's emotions while they are typing, detecting tension and stress. When the user clicks the save button after the meeting ends, the terminal sends the meeting minutes and emotion data to the server, which stores them in a central database.

[0496] Planning and report writing

[0497] Users create drafts of projects and reports using a terminal. The terminal provides users with necessary templates and past materials, and the emotion engine analyzes whether the user is experiencing stress or satisfaction. Completed projects and reports are sent to a server, stored in a central database, and notifications are sent to relevant parties. The emotion engine provides feedback tailored to the user's emotions.

[0498] Market research

[0499] The user sets market research questions on their device and sends the set questions to the server. The server generates a survey link and sends the link to each research participant based on the distribution list. The emotion engine analyzes the user's response during distribution. Research participants answer the survey via the link, and the data is sent from their device to the server. The server analyzes the collected response data and generates a visual report. The emotion engine uses this aggregated data to analyze emotional trends and reflects them in the results.

[0500] Creating a promotional video

[0501] The user uses a device to plan the video content and shooting schedule, and an emotion engine analyzes the user's emotions and incorporates them into the video content. The device then uses video editing software to edit the footage and sends the final version to the server. The server stores the completed video, generates the necessary links, and provides them to the user.

[0502] System testing and monitoring

[0503] The terminal executes a test script and sends the results to the server. The server collects operation logs from each terminal and monitors for anomalies. The sentiment engine evaluates the user's emotions while using the system and optimizes the user experience.

[0504] Call center initial support

[0505] Operators input customer inquiries into a terminal, and an emotion engine analyzes the operator's emotions during the interaction, providing stress reduction advice. The server stores the inquiry details in a central database and combines them with emotion data to evaluate the quality of customer service.

[0506] Examples of specific cases and prompt statements

[0507] For example, when a user uses the "market research" function, the process proceeds as follows:

[0508] 1. Users input market research questions from their devices, and an emotion engine analyzes the user's facial expressions and voice as they input the information.

[0509] 2. The terminal sends the entered questions to the server, and the server stores the questions and sentiment data in a central database.

[0510] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[0511] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[0512] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[0513] Example of a prompt:

[0514] "Based on the analysis of sentiment data from the following market research questions, please tell us what improvements can be made."

[0515] In this way, the system helps users conduct market research effectively and provides higher quality services by utilizing sentiment data.

[0516] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0517] Step 1: Enter schedule

[0518] 1.1 Users input their daily schedules and tasks from their respective devices. Input includes items such as "meetings," "document creation," and "lunch meetings."

[0519] 1.2 The terminal temporarily saves the entered schedule information. The input is the schedule information entered by the user.

[0520] 1.3 The emotion engine analyzes the user's facial expressions and voice during input and generates emotion data. Input data comes from the user's camera and microphone.

[0521] 1.4 The terminal sends schedule information and sentiment data to the server. The output includes schedule information and sentiment data.

[0522] 1.5 The server saves schedule information and sentiment data to the central database. Schedule information and sentiment data are saved to the central database as output.

[0523] Step 2: Schedule synchronization

[0524] 2.1 The server synchronizes the latest schedule information and sentiment feedback to each terminal. The input is information stored in a central database, and the output is updated data sent to the terminal.

[0525] 2.2 The device locally stores the schedule information and sentiment feedback it receives. A specific example is push notifications for a scheduling app.

[0526] Step 3: Enter meeting minutes

[0527] 3.1 Users enter meeting minutes into their terminals during the meeting. This input includes information such as "agenda," "decisions," and "next actions."

[0528] 3.2 The terminal temporarily saves meeting minutes in real time. The input is the meeting minutes information entered by the user, and the output is the temporarily saved data.

[0529] 3.3 The emotion engine recognizes the user's emotions during input and detects tension and stress. The user's facial expression data is the input, and emotion data is generated as the output.

[0530] Step 4: Send and save meeting minutes

[0531] 4.1 The user clicks the save button after the meeting ends. User actions are included as input.

[0532] 4.2 The terminal sends meeting minutes and sentiment data to the server. Meeting minutes and sentiment data are sent to the server as output.

[0533] 4.3 The server saves the received meeting minutes and sentiment data to the central database. The output includes the meeting minutes and sentiment data stored in the central database.

[0534] Step 5: Planning and Report Creation

[0535] 5.1 Users create drafts of plans and reports using their devices. The input includes the content of the plan.

[0536] 5.2 The terminal provides users with necessary templates and historical documents. Related documents are displayed to the user as output.

[0537] 5.3 The emotion engine analyzes whether the user is experiencing stress or satisfaction. Inputs include the user's keystroke patterns and mouse movements, and emotional data is generated as output.

[0538] 5.4 The terminal sends the completed project proposal or report to the server. The project proposal or report is sent to the server as output.

[0539] 5.5 The server stores them in a central database and sends notifications to the relevant parties. The output includes the stored data and the sent notifications.

[0540] 5.6 The emotion engine provides feedback that corresponds to the user's emotions. The feedback is displayed to the user as output.

[0541] Step 6: Market Research

[0542] 6.1 The user sets the market research questions on the device. The questions are included as input.

[0543] 6.2 The terminal sends the configured questions to the server. The question items are sent to the server as output.

[0544] 6.3 The server generates the survey link and sends the link to each survey participant based on the distribution list. The generated link and the link itself are sent as output.

[0545] 6.4 The emotion engine analyzes the user's reactions. User facial expression data is taken as input, and analyzed emotion data is generated as output.

[0546] 6.5 Participants answer the questionnaire via a link, and their devices send the data to the server. The response data is included as input.

[0547] 6.6 The server stores the collected response data, and the sentiment engine analyzes the sentiment trends. The output includes the stored response data and the analysis results.

[0548] Step 7: Create a promotional video

[0549] 7.1 The user uses a device to formulate the video content and shooting plan. The video content is included as input.

[0550] 7.2 The emotion engine analyzes the user's emotions and reflects them in the video content. User emotion data is the input, and it is reflected in the video content as the output.

[0551] 7.3 The terminal uses video editing software to edit the footage and sends the final version to the server. The edited video is sent to the server as output.

[0552] 7.4 The server stores the completed video and generates the necessary links to provide to the user. The links are generated and provided as output.

[0553] Step 8: System Testing and Monitoring

[0554] 8.1 The terminal executes the test script and sends the results to the server. The test results are included as input.

[0555] 8.2 The server collects operation logs from each terminal and monitors for anomalies. Operation logs are included as input.

[0556] 8.3 The emotion engine evaluates the user's emotions while using the system and optimizes the user experience. The input is user emotion data, and the output is optimized feedback.

[0557] Step 9: Temporary support at the call center

[0558] 9.1 An operator enters a customer inquiry into a terminal, and the emotion engine analyzes the response. The input includes the content of the inquiry.

[0559] 9.2 The terminal sends the entered query content to the server. The query data is sent to the server as output.

[0560] 9.3 The server stores the inquiry details in a central database and evaluates the quality of customer service by combining them with sentiment data. The output includes the stored data and the evaluation results.

[0561] (Application Example 2)

[0562] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0563] In modern brick-and-mortar stores, much effort is being made to improve customer satisfaction, and as part of this, it is necessary to accurately understand customer emotions and provide services based on those emotions. However, with previous systems, it was difficult to recognize customer emotions in real time and provide optimal feedback based on that information. Furthermore, it was difficult to centrally manage customer emotion data and use it to help with future visits. As a result, customer satisfaction tended to decline and the quality of service varied.

[0564] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and save it in a central database; means for users to input meeting minutes during a meeting and send them to the server after the meeting ends; means for users to create plans and reports, save them after completion, and notify relevant parties; means for users to set market research questions, distribute questionnaires, and collect and analyze response data; means for managing employee training curricula, distributing training modules, and reporting progress; means for users to formulate video content and shooting plans, edit them, and save the final version to the server; means for collecting system operation logs, detecting and reporting problems; means for inputting and recording inquiry content to call center operators; means for recognizing customer emotional information in real time and providing store staff with the most appropriate service according to that emotional state; and means for saving past visit data and emotional history and providing the most appropriate service on the next visit. This makes it possible to recognize customer emotional states in real time and provide the most appropriate service based on that information, thereby improving customer satisfaction and standardizing service quality.

[0565] A "user" refers to a person who uses a system to input, manipulate, and manage various types of information.

[0566] A "terminal" refers to a device used by a user to input and manipulate information. Examples include computers, smartphones, and tablets.

[0567] A "server" refers to a central system that receives, stores, and processes information sent by users.

[0568] A "central database" refers to a large-scale database where information managed by a system is stored centrally.

[0569] An "emotion engine" refers to a system that analyzes a user's facial expressions, voice, and behavioral patterns to recognize their emotions in real time.

[0570] "Schedule information" refers to information about appointments and tasks entered by the user.

[0571] "Meeting" refers to a conference or meeting where users enter meeting minutes.

[0572] "Meeting minutes" refers to a document that records the content discussed and decisions made during a meeting.

[0573] "Planning" refers to a plan or concept for achieving a specific objective.

[0574] A "report" refers to a document that summarizes the results of research and analysis on a specific theme or topic.

[0575] "Market research" refers to research activities conducted to understand consumer needs and market trends.

[0576] A "survey" refers to a method used in market research to distribute questions to target individuals and collect their responses.

[0577] "Employee training" refers to educational programs implemented with the aim of improving employees' skills and knowledge.

[0578] An "educational module" refers to an individual learning unit used for employee training.

[0579] "Video content" refers to information or plans that are recorded as video.

[0580] A "filming plan" refers to the plan and schedule for shooting the content of a video.

[0581] "Operation log" refers to the operation history and event records of a system or terminal.

[0582] A "call center" refers to a department that has a customer service function, handling inquiries and providing support.

[0583] An "operator" refers to a person who handles customer inquiries in a call center.

[0584] A "customer" refers to a person who visits a physical store and receives services.

[0585] "Visit data" refers to information about when a customer visits a store.

[0586] "Service" refers to the benefits and assistance that a store provides to its customers.

[0587] The following describes the specific mechanism and processing of the system as an embodiment of this invention. This system is realized through the collaboration of a user, a terminal, a server, and an emotion engine.

[0588] Program generation and processing flow

[0589] Hardware and software usage

[0590] Hardware: User devices include smartphones, tablets, smart glasses, and head-mounted displays. Terminals used by store staff, as well as cameras and sensors for analyzing emotional data, will also be used.

[0591] Software: Python, OpenCV, and an emotion recognition library (e.g., EmotionRecognition) are used. This allows for real-time recognition of customer facial expressions and voices by analyzing camera footage, and generates emotion data.

[0592] User

[0593] Users input various data such as schedule information, meeting minutes, plans, reports, and survey settings through their devices and send it to the server. In physical stores, emotional information of users when they visit as customers is also collected. For example, store staff using smart glasses scan customers' facial expressions and analyze that data.

[0594] terminal

[0595] The terminal temporarily stores user-entered data and sends it to the server. Furthermore, it analyzes emotional data collected through the emotion engine, monitors the results in real time, and provides appropriate feedback. In physical stores, it analyzes camera footage to understand customer emotions. It also manages past visit data and emotional history to provide optimal service for subsequent visits.

[0596] server

[0597] The server stores schedule information, meeting minutes, plans, reports, survey responses, training module progress, activity logs, and sentiment data submitted by users in a central database. It also synchronizes the latest information with each terminal and provides necessary notifications. In physical stores, it analyzes customer sentiment data in real time and supports the provision of optimal services based on the results.

[0598] Specific example

[0599] Consider a scenario where store staff at a physical store use smart glasses to analyze customers' emotions in real time. If the staff member detects that the customer is stressed, a notification will appear on their device saying, "Please provide a relaxing service." Furthermore, if there is data indicating that the customer has previously been relaxed, the system will suggest the most suitable service for that customer.

[0600] Example of a prompt

[0601] "We are developing an application that uses new technology to recognize customer emotions in real time and provide optimal feedback to store staff. Specifically, it's a system that analyzes camera footage to predict customer emotions and notifies staff of the results. Could you please provide a detailed code example for this system?"

[0602] Thus, this invention aims to improve customer service in physical stores by working in conjunction with an emotion engine.

[0603] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0604] Step 1:

[0605] The user enters schedule information into the terminal. The terminal temporarily stores the entered schedule information and sends it to the server. This schedule information includes the date, time, and details of the appointment. The entered data is temporarily stored on the terminal and sent to the server.

[0606] Step 2:

[0607] The server receives schedule information sent by users and stores it in a central database. The server analyzes the received schedule information and synchronizes the latest schedule information to each terminal. This ensures that all devices have the most up-to-date schedule information.

[0608] Step 3:

[0609] During a meeting, the user enters meeting minutes into a terminal. The terminal temporarily stores the entered minutes in real time and sends them to the server after the meeting ends. At this stage, the emotion engine analyzes the user's facial expressions and voice to generate emotion data. The meeting minutes and emotion data are temporarily stored on the terminal.

[0610] Step 4:

[0611] The server receives meeting minutes and sentiment data after meetings and stores them in a central database. The server then analyzes the stored data and generates necessary feedback. This process manages both the meeting minutes and sentiment data.

[0612] Step 5:

[0613] Users input plans and reports into a terminal, and upon completion, save them and notify relevant parties. The terminal temporarily stores drafts of the plans and reports, and sends them to the server upon completion. Even at this stage, the emotion engine monitors the user's state and analyzes stress and satisfaction levels.

[0614] Step 6:

[0615] The server receives the final versions of proposals and reports, along with sentiment data, and stores them in a central database. The server analyzes the received data and sends notifications to relevant parties. This facilitates the management of proposals and reports and the provision of sentiment feedback.

[0616] Step 7:

[0617] The user enters market research questions into a terminal and sends them to the server. When creating the survey, the emotion engine analyzes the user's state and generates emotion data. The server receives and stores the questions and emotion data.

[0618] Step 8:

[0619] The server generates a survey link and sends it to each participant based on the distribution list. During this sending process, the sentiment engine analyzes user responses. The collected survey response data is temporarily stored on the server.

[0620] Step 9:

[0621] The user enters their video shooting plan into the terminal and sends the completed video to the server. The terminal edits the video and temporarily saves the final version. During shooting, an emotion engine monitors the user's state and analyzes emotional data.

[0622] Step 10:

[0623] The server receives the completed video and sentiment data and stores them in a central database. The server generates the necessary links and provides them to the user. This allows for the management of video content and sentiment feedback.

[0624] Step 11:

[0625] Store staff use smart glasses to scan customers' facial expressions. The device collects customer facial data in real time, and an emotion engine analyzes it to generate emotion data. Real-time emotion information of the customer when they visit the store is displayed on the device.

[0626] Step 12:

[0627] The server receives customer sentiment data and stores it in a central database along with past visit data and sentiment history. Based on the analysis results, the server sends notifications to store staff to provide the best possible service. This improves the customer experience.

[0628] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0629] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0630] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0631] [Second Embodiment]

[0632] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0633] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0634] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0635] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0636] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0637] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0638] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0639] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0640] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0641] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0642] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0643] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0644] This invention relates to a system for centrally managing and efficiently executing various tasks. This system enables the efficient operation of a wide range of tasks through the collaboration of users, terminals, and servers.

[0645] Daily schedule management

[0646] 1. Schedule entry:

[0647] Users enter their daily schedules and tasks from each device.

[0648] The terminal temporarily saves the entered schedule and sends it to the server.

[0649] 2. Schedule saving and synchronization:

[0650] The server stores the received schedule information in a central database.

[0651] The server synchronizes the latest schedule information to each terminal after saving it.

[0652] Meeting minutes

[0653] 1. Entering meeting minutes:

[0654] The user enters meeting minutes into their terminal during the meeting.

[0655] The terminal temporarily saves meeting minutes in real time.

[0656] 2. Sending and saving meeting minutes:

[0657] The user clicks the save button after the meeting ends.

[0658] The terminal sends the meeting minutes to the server.

[0659] The server saves the received meeting minutes to a central database.

[0660] Planning and report writing

[0661] 1. Planning and input:

[0662] Users create drafts of plans and reports using their devices.

[0663] The terminal provides users with necessary templates and historical documents.

[0664] 2. Saving and Notifications:

[0665] The terminal sends completed plans and reports to the server.

[0666] The server stores them in a central database and sends notifications to the relevant parties.

[0667] Market research

[0668] 1. Creating and preparing the questionnaire:

[0669] The user sets the market research questions on their device.

[0670] The terminal sends the configured question to the server.

[0671] 2. Distribution of questionnaires:

[0672] The server generates a survey link and sends it to each recipient based on the distribution list.

[0673] 3. Response collection and analysis:

[0674] Participants will answer the questionnaire via the link they receive.

[0675] The device sends the response data to the server.

[0676] The server analyzes the collected response data, generates a visual report, and provides it to the user.

[0677] Employee training

[0678] 1. Curriculum management and delivery:

[0679] The server manages the employee training curriculum and distributes training modules to each terminal.

[0680] 2. Report on educational progress:

[0681] The terminal reports the progress of the educational module to the server in real time.

[0682] Users enter feedback after each session is completed and send it to the server.

[0683] Creating a promotional video

[0684] 1. Planning and developing video content:

[0685] The user uses the device to plan the video content and shooting schedule.

[0686] 2. Filming and editing:

[0687] The terminal uses video editing software to edit the footage and sends the final version to the server.

[0688] The server stores the completed video and generates the necessary links to provide to the user.

[0689] System testing and monitoring

[0690] 1. Run the test:

[0691] The terminal executes the test script and sends the results to the server.

[0692] 2. Log collection and monitoring:

[0693] The server collects operation logs from each terminal and monitors for anomalies.

[0694] The server will report any problems detected to the user.

[0695] Call center initial support

[0696] 1. Inquiry handling:

[0697] The operator enters the customer's inquiry into the terminal.

[0698] 2. Recording and Analysis:

[0699] The server stores the query content in a central database and performs analysis.

[0700] Specific example:

[0701] For example, when a user uses the "market research" function, the process proceeds as follows:

[0702] 1. The user sets the survey questions on their device and submits them.

[0703] 2. The server receives the questions and generates a survey link.

[0704] 3. The device sends the link to the person being surveyed.

[0705] 4. Participants will answer the questionnaire using the provided link.

[0706] 5. The device sends the response data to the server.

[0707] 6. The server aggregates the data, generates a report including the analysis results, and provides it to the user.

[0708] This allows the system to help users conduct market research effectively.

[0709] The following describes the processing flow.

[0710] Market research processing steps

[0711] Creating and preparing the questionnaire for distribution

[0712] Step 1:

[0713] The user enters market research survey questions from their device.

[0714] The terminal temporarily stores the entered questions in local storage and checks the integrity of the input.

[0715] Step 2:

[0716] The terminal sends the questions that have completed the integrity check to the server.

[0717] The server reviews the received questions and saves them to the central database.

[0718] Distribution of questionnaires

[0719] Step 3:

[0720] The server generates a link to the survey and emails the link to each survey participant based on the distribution list.

[0721] Step 4:

[0722] The terminal reports the transmission status to the server in real time.

[0723] The server confirms that the survey link has been successfully sent and updates the log.

[0724] Collection of responses

[0725] Step 5:

[0726] Participants click the received link to access the survey page.

[0727] The device displays the survey response screen, and the survey participants answer the questions.

[0728] Step 6:

[0729] The device temporarily stores the survey participants' responses in local storage and sends the data to the server once all responses have been completed.

[0730] Step 7:

[0731] The server saves the received response data to a central database and returns a confirmation message to the terminal acknowledging the successful saving.

[0732] Data aggregation and analysis

[0733] Step 8:

[0734] The server aggregates the stored response data and performs statistical analysis.

[0735] The server converts the analysis results into graphs and charts, generating visual reports.

[0736] Step 9:

[0737] The server sends the generated report to the terminal.

[0738] Users can view reports from their devices and provide feedback as needed.

[0739] Record of feedback

[0740] Step 10:

[0741] The device sends the user's input to the server.

[0742] The server saves the received feedback to a database and records it as an area for improvement in future market research.

[0743] The above processing steps enable the system to efficiently carry out the entire market research process, from data collection and analysis to reporting, in a consistent manner.

[0744] (Example 1)

[0745] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0746] Currently, there is a need for a centralized management system to efficiently handle multiple functions in business management and operations, such as schedule management, meeting minute creation, planning and report creation, market research, employee training, promotional video production, system testing and monitoring, and call center inquiry handling. When these tasks are managed individually, data synchronization and sharing become cumbersome, leading to decreased efficiency. Furthermore, there is a growing need for a system that reduces user effort while enabling accurate and rapid data processing.

[0747] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0748] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for terminals to temporarily store schedules and send them to the server, and means for the server to store schedule information in a central database and synchronize it with each terminal. This enables centralized management of schedule information and real-time data synchronization. It also includes means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to create plans and reports, save them after completion and notify relevant parties, and means for users to set market research questions, distribute questionnaires, and collect and analyze response data. The server also includes means for generating questionnaire links and sending them to each target based on a distribution list, means for managing employee training curricula, distributing training modules and reporting progress, means for users to formulate and edit video content and shooting plans and save the final version to the server, means for the server to collect system operation logs, detect and report problems, and means for call center operators to input and record inquiry details. This enables efficient and accurate centralized management of multiple tasks, significantly reducing user effort and improving overall operational efficiency.

[0749] A "user" refers to an entity that uses the system to perform various tasks.

[0750] A "terminal" refers to a device operated by a user, such as a computer or smartphone used for tasks like entering schedules, creating meeting minutes, or designing surveys.

[0751] A "server" refers to a central computer system used to store data and manage communication and data synchronization between terminals.

[0752] A "central database" refers to a database built on a server that centrally manages and stores all data generated within the system.

[0753] "Schedule information" refers to data about appointments and tasks entered by the user.

[0754] "Meeting minutes" refers to a document that users record during a meeting, including details of the meeting, discussions, and decisions made.

[0755] "Plans" and "reports" refer to plans and reports created by users, which are documents containing information about the business or project.

[0756] "Market research" refers to the process of collecting market reactions and opinions through surveys set by users.

[0757] A "survey link" is a URL generated by the server and sent to the survey participant, used to access the survey form.

[0758] An "employee training curriculum" refers to an educational program aimed at improving employees' skills and acquiring knowledge.

[0759] An "education module" is a part of the employee training curriculum and refers to the specific content of each lesson or session.

[0760] "Video content" refers to the scenario and structure that the user develops when creating a promotional video.

[0761] A "shooting plan" refers to a specific plan for shooting a video, including the shooting schedule, locations, and necessary equipment.

[0762] "Operation log" refers to data that records the operating status of a system or terminal.

[0763] A "call center operator" refers to a person who handles customer inquiries at a call center.

[0764] This invention relates to a system for centrally managing and efficiently executing various tasks. Through collaboration between users, terminals, and servers, it enables the efficient operation of a wide range of tasks.

[0765] Hardware and software to be used

[0766] hardware

[0767] Devices: Laptops, smartphones (iOS / Android), tablets (iPad), desktop PCs

[0768] Server: Cloud servers (AWS, Azure, Google Cloud)

[0769] software

[0770] Database management systems (PostgreSQL, MySQL)

[0771] Text editor

[0772] Video editing software

[0773] Communication method (Firebase Cloud Messaging, HTTPS)

[0774] Email sending system (SendGrid)

[0775] Analysis tools (Python Pandas, Matplotlib)

[0776] The specific embodiments of each function of this system are described below.

[0777] Schedule management

[0778] Input and data transmission

[0779] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[0780] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[0781] Data storage and synchronization

[0782] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[0783] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[0784] Market research

[0785] Questionnaire creation and distribution

[0786] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[0787] The terminal POSTs the question data to the server in JSON format.

[0788] Generate and send survey link

[0789] The server uses a URL shortening service to generate the survey link.

[0790] The server will send this link to the recipients using the email sending system (SendGrid).

[0791] Response collection and analysis

[0792] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[0793] The device temporarily stores the response data and sends it to the server in JSON format.

[0794] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[0795] Example of a prompt

[0796] Schedule management:

[0797] "Please add a 10:00 meeting to this week's schedule and save it to the server."

[0798] Market research:

[0799] "Please create a questionnaire about the new product and send it to the target audience."

[0800] The various functions of this system allow users to efficiently centralize and manage multiple tasks, significantly reducing effort and improving overall operational efficiency.

[0801] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0802] Daily schedule management

[0803] Step 1:

[0804] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[0805] Input: Date, time, meeting content (e.g., 10:00, meeting with a client)

[0806] Action: The user creates a new schedule in the calendar app and clicks the save button.

[0807] Output: Schedule data stored in the local cache

[0808] Step 2:

[0809] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[0810] Input: Schedule data (e.g., 10:00, Meeting with a client)

[0811] Operation: Save to local cache and convert to JSON format.

[0812] Output: POST request to the server

[0813] Step 3:

[0814] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[0815] Input: Schedule data in JSON format

[0816] Operation: Data analysis and insertion into a PostgreSQL database.

[0817] Output: Schedule data stored in the database

[0818] Step 4:

[0819] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[0820] Input: Schedule data stored in the database

[0821] Operation: Triggering push notifications using Firebase Cloud Messaging

[0822] Output: Latest schedule notification received on each device

[0823] Market research

[0824] Step 1:

[0825] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[0826] Input: Question item (e.g., What do you think of the product?)

[0827] Operation: Enter and save a new question on the web interface.

[0828] Output: Question data stored in the local cache

[0829] Step 2:

[0830] The terminal POSTs the question data to the server in JSON format.

[0831] Input: Question data (e.g., What do you think of the product?)

[0832] Operation: Save to local cache and convert to JSON format.

[0833] Output: POST request to the server

[0834] Step 3:

[0835] The server uses a URL shortening service to generate the survey link.

[0836] Input: Question data

[0837] Operation: Generates survey links using a URL shortening service.

[0838] Output: Generated survey link

[0839] Step 4:

[0840] The server will send this link to the recipients using the email sending system (SendGrid).

[0841] Input: Generated survey link

[0842] Operation: Send emails based on distribution lists using SendGrid.

[0843] Output: Email and survey link sent to participants

[0844] Step 5:

[0845] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[0846] Input: Response data entered by survey participants.

[0847] How it works: Fill out the survey form in your web browser and submit your response.

[0848] Output: Sending response data to the server

[0849] Step 6:

[0850] The device temporarily stores the response data and sends it to the server in JSON format.

[0851] Input: Response data

[0852] Operation: Save to local cache and convert to JSON format.

[0853] Output: POST request to the server

[0854] Step 7:

[0855] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[0856] Input: Response data in JSON format

[0857] Function: Saves to database, runs analytical algorithms, generates visual reports (Pandas, Matplotlib)

[0858] Output: Visual reports displayed on the user's dashboard.

[0859] Example of a prompt

[0860] Schedule Management: "Add a 10:00 meeting to this week's schedule and save it to the server."

[0861] Market research: "Create a questionnaire about the new product and send it to the target audience."

[0862] This program's processing steps allow users to efficiently manage schedules and conduct market research, enabling them to manage various tasks in an integrated manner.

[0863] (Application Example 1)

[0864] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0865] Traditional business management systems have struggled to efficiently integrate scheduling, real-time tracking of operations, meeting minute creation, employee training, and system testing and monitoring within logistics centers. In particular, the inability to track the movement of goods and the progress of picking lists within the logistics center in real time led to decreased operational efficiency and increased errors. A new system is needed to address these challenges.

[0866] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0867] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to set market research questions, distribute questionnaires, and collect and analyze response data, and means for tracking the movement of goods within the logistics center and the progress of picking lists in real time. This enables efficient operation of the entire logistics center and real-time management of various operations.

[0868] A "user" is an entity that uses a system to perform various tasks.

[0869] A "terminal" is a device used by a user to input information or utilize system functions.

[0870] A "central database" is a storage device used to centrally store information for the entire system.

[0871] A "server" is a computer system that receives and processes data transmitted from various terminals.

[0872] "Schedule information" refers to the daily schedules and tasks entered by the user.

[0873] "Meeting minutes" are documents that users record during a meeting and save afterward.

[0874] A "project" is a new project or plan created by a user.

[0875] A "report" is a document that summarizes the progress and results of a project.

[0876] "Market research" is the activity of collecting and analyzing data about a specific market.

[0877] A "questionnaire" is a survey conducted as part of market research, consisting of a series of questions.

[0878] "Data collection" refers to the act of gathering information transmitted from each device.

[0879] "Data analysis" is the process of verifying collected information and compiling it into a visual report.

[0880] An "employee training curriculum" is a learning program aimed at improving employees' skills.

[0881] An "education module" refers to software and teaching materials used to implement employee training curricula.

[0882] "Feedback" refers to the evaluations and opinions that users enter after completing an educational module.

[0883] A "promotional video" is video content produced for the purpose of advertising or public relations.

[0884] "Real-time tracking" is a function that allows you to instantly grasp the status of ongoing operations and logistics.

[0885] A "picking list" is a list that displays the items needed for a specific order or task.

[0886] This invention is a system for efficiently managing and operating operations at a logistics center. The specific program and its processing details for realizing this system are described below.

[0887] System Configuration

[0888] This system consists primarily of users, production terminals, an information processing server, and a central database. The terminals provide an interface for users to input information and send it to the server. The server stores the received information in the central database and synchronizes it with other terminals as needed.

[0889] Schedule management

[0890] Users input their daily tasks and schedules within the logistics center using terminals. This schedule information is temporarily stored on the terminals and then sent to the server. The server stores the received schedule information in a central database and synchronizes the latest schedule information with each terminal.

[0891] Real-time tracking

[0892] Data is entered via terminals to track the movement of goods within the logistics center and the progress of picking lists in real time. The terminals transmit this information to a server in real time, and the server stores it in a central database, thereby improving the efficiency of goods management.

[0893] Meeting minutes

[0894] During the meeting, users enter meeting minutes into their terminals. After the meeting ends, users click the save button, and the minutes are sent from their terminals to the server. The server saves the received minutes to a central database and sends notifications to the relevant parties.

[0895] Employee training

[0896] The server manages the employee training curriculum and delivers training modules to each terminal. Training progress reports are sent from the terminal to the server in real time. Users enter feedback after completing each session and send it to the server.

[0897] System testing and monitoring

[0898] The terminal collects system operation logs and sends them to the server. The server stores the operation logs of each terminal in a central database and reports any abnormalities to the user.

[0899] Specific example

[0900] For example, to add a new inventory check task, the user would enter the following information on the schedule input screen:

[0901] Example of a schedule input prompt

[0902] Please add a new task.

[0903] User: employee1

[0904] Task name: Inventory check

[0905] Date: 2023-10-10

[0906] Time: 14:00

[0907] In this way, the system enables efficient operation of the entire logistics center and real-time management of various tasks. Furthermore, it can streamline other incidental tasks such as schedule management and meeting minute creation.

[0908] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0909] Step 1:

[0910] The user opens the schedule entry screen using their device and enters their daily tasks and appointments. The entered data includes the username, task name, date, and time. The device temporarily saves the entered data and then sends it to the server.

[0911] Step 2:

[0912] The server receives schedule information sent from the terminals. The data includes the username, task name, date, and time, and this data is stored in a central database. After the schedule information is stored, the server sends a request to each terminal to synchronize with the latest schedule information.

[0913] Step 3:

[0914] Users use terminals to input information about the movement of items within the logistics center and picking lists in real time. The entered data is immediately sent from the terminal to the server.

[0915] Step 4:

[0916] The server receives real-time tracking data sent from the terminal and stores it in a central database. Based on the received data, the server monitors the movement of items and the progress of picking lists in real time.

[0917] Step 5:

[0918] During a meeting, the user enters meeting minutes using their device. After the meeting ends, the user clicks the save button on their device. The device temporarily saves the meeting minutes and sends them to the server.

[0919] Step 6:

[0920] The server receives meeting minutes data sent from terminals and stores it in a central database. This data includes the meeting title, agenda, participants, and minutes content. The server notifies relevant parties that the minutes have been saved.

[0921] Step 7:

[0922] Users access the employee training curriculum from their devices and utilize the training modules. After completing each session, users provide feedback. The device then sends the feedback and progress report to the server.

[0923] Step 8:

[0924] The server receives progress data and feedback on educational modules sent from terminals and stores them in a central database. The server manages educational progress and feedback in real time.

[0925] Step 9:

[0926] The terminal collects system operation status as logs and periodically sends them to the server. The log data includes operational anomalies, error messages, and performance metrics.

[0927] Step 10:

[0928] The server receives operation logs sent from the terminal and stores them in a central database. The server analyzes the log data and issues a warning to the user if an anomaly is detected.

[0929] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0930] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[0931] Emotional engine integration

[0932] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[0933] Daily schedule management

[0934] 1. Schedule entry:

[0935] Users enter their daily schedules and tasks from each device.

[0936] The terminal temporarily stores the entered schedule information and sends it to the server.

[0937] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data, which is then sent to the server.

[0938] 2. Schedule saving and synchronization:

[0939] The server stores schedule information and sentiment data in a central database.

[0940] The server synchronizes the latest schedule information and emotional feedback to each terminal.

[0941] Meeting minutes

[0942] 1. Entering meeting minutes:

[0943] The user enters meeting minutes into their terminal during the meeting.

[0944] The terminal temporarily saves meeting minutes in real time.

[0945] The emotion engine recognizes the user's emotions while they are typing and detects tension and stress.

[0946] 2. Sending and saving meeting minutes:

[0947] The user clicks the save button after the meeting ends.

[0948] The terminal sends meeting minutes and sentiment data to the server.

[0949] The server stores the received meeting minutes and sentiment data in a central database.

[0950] Planning and report writing

[0951] 1. Planning and input:

[0952] Users create drafts of plans and reports using their devices.

[0953] The terminal provides users with necessary templates and historical documents.

[0954] The emotion engine analyzes whether the user is experiencing stress or satisfaction.

[0955] 2. Saving and Notifications:

[0956] The terminal sends completed plans and reports to the server.

[0957] The server stores them in a central database and sends notifications to the relevant parties.

[0958] The emotion engine provides feedback that responds to the user's emotions.

[0959] Market research

[0960] 1. Creating and preparing the questionnaire:

[0961] The user sets the market research questions on their device.

[0962] The terminal sends the configured question to the server.

[0963] 2. Distribution of questionnaires:

[0964] The server generates the survey link and sends the link to each survey participant based on the distribution list.

[0965] The emotion engine analyzes the user's reaction when a link is sent.

[0966] 3. Response collection and analysis:

[0967] Participants click the received link to access the survey page.

[0968] The device displays the survey response screen, and the survey participants answer the questions.

[0969] The device sends the response data to the server.

[0970] The server analyzes the collected response data and generates a visual report.

[0971] The emotion engine analyzes emotional tendencies based on aggregated data and reflects them in the results.

[0972] Employee training

[0973] 1. Curriculum management and delivery:

[0974] The server manages the employee training curriculum and distributes training modules to each terminal.

[0975] The emotion engine adjusts the curriculum based on employee emotional data.

[0976] 2. Report on educational progress:

[0977] The terminal reports the progress of the educational module to the server in real time.

[0978] Users enter feedback after each session is completed and send it to the server.

[0979] The server receives feedback and sentiment data and analyzes the educational effect.

[0980] Creating a promotional video

[0981] 1. Planning and developing video content:

[0982] The user uses the device to plan the video content and shooting schedule.

[0983] The emotion engine analyzes the user's emotions and reflects them in the video content.

[0984] 2. Filming and editing:

[0985] The terminal uses video editing software to edit the footage and sends the final version to the server.

[0986] The server stores the completed video and generates the necessary links to provide to the user.

[0987] System testing and monitoring

[0988] 1. Run the test:

[0989] The terminal executes the test script and sends the results to the server.

[0990] 2. Log collection and monitoring:

[0991] The server collects operation logs from each terminal and monitors for anomalies.

[0992] The emotion engine evaluates the user's emotions while using the system and optimizes the user experience.

[0993] Call center initial support

[0994] 1. Inquiry handling:

[0995] The operator enters the customer's inquiry into the terminal.

[0996] The emotion engine analyzes the operator's emotions during interactions and provides advice for stress reduction.

[0997] 2. Recording and Analysis:

[0998] The server stores the query content in a central database and performs analysis.

[0999] The emotion engine combines collected data with emotional data to evaluate the quality of customer service.

[1000] Specific example:

[1001] For example, when a user uses the "market research" function, the process proceeds as follows:

[1002] 1. The user inputs market research questions from their device. The emotion engine analyzes the user's facial expressions and voice as they input the information.

[1003] 2. The terminal sends the entered questions to the server. The server stores the questions and sentiment data in a central database.

[1004] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[1005] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[1006] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[1007] This allows the system to help users conduct market research effectively and to provide higher quality services by utilizing sentiment data.

[1008] The following describes the processing flow.

[1009] Processing steps in market research

[1010] Creating and preparing the questionnaire for distribution

[1011] Step 1:

[1012] The user enters market research questions from their terminal.

[1013] The device temporarily stores the entered questions and sends them to the emotion engine.

[1014] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data.

[1015] The emotion engine sends emotional data back to the device.

[1016] Step 2:

[1017] The device sends the questionnaire items and sentiment data to the server.

[1018] The server stores the questionnaire items and sentiment data in a central database.

[1019] Distribution of questionnaires

[1020] Step 3:

[1021] The server generates a survey link and sends the link via email to each survey participant based on the distribution list.

[1022] Step 4:

[1023] The terminal reports the transmission status to the server in real time.

[1024] The server logs the successful submission of the survey link.

[1025] The emotion engine re-analyzes the user's response during this process and updates the emotion data.

[1026] Collection of responses

[1027] Step 5:

[1028] Participants in the survey will access the questionnaire page by clicking the link they receive.

[1029] The device displays the survey response screen, and the survey participants answer the questions.

[1030] Step 6:

[1031] The device temporarily stores the survey participants' responses and sends the data to the server once all responses have been completed.

[1032] Step 7:

[1033] The server stores the response data in a central database.

[1034] The emotion engine analyzes the emotional data of the survey participants on the server and stores it together with the data in the database.

[1035] The server sends a confirmation message to the terminal acknowledging that the save was successful.

[1036] Data aggregation and analysis

[1037] Step 8:

[1038] The server aggregates the stored response data and performs statistical analysis.

[1039] The emotion engine works with the server to analyze emotional tendencies based on aggregated data and reflect them in the results.

[1040] Step 9:

[1041] The server converts the analysis results into graphs and charts, generating visual reports.

[1042] The server sends the generated report and sentiment analysis results to the terminal.

[1043] Step 10:

[1044] Users can view reports from their devices and provide feedback as needed.

[1045] The device sends user feedback to the emotion engine.

[1046] The emotion engine analyzes feedback and emotional data to derive areas for improvement for the next market research.

[1047] The emotion engine sends the analysis results to the server and stores them in the database.

[1048] This allows the system to efficiently manage the entire market research process and leverage user sentiment data to provide high-quality feedback and analytical results.

[1049] (Example 2)

[1050] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1051] Traditional business management systems focus on individual tasks and schedule management, but lack mechanisms to provide feedback that takes into account the user's emotional state. Furthermore, few systems utilize emotional data to improve work efficiency and quality. As a result, there is no effective method for managing user stress and satisfaction, making it a challenge to improve work quality and employee mental health.

[1052] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database along with emotional data; means for users to input meeting minutes during a meeting, perform emotional analysis, and send them to the server after the meeting ends; and means for users to create plans and reports, receive emotional feedback, and then save them and notify relevant parties after completion. This enables the analysis of users' emotional data in real time and provides optimal feedback based on that analysis, thereby improving work efficiency and the mental health of employees.

[1053] "Schedule information" refers to appointments and action plans that users enter to manage their daily work and tasks.

[1054] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions, voice, behavioral patterns, and other factors.

[1055] A "central database" is a database used to centrally manage and store data for the entire system.

[1056] Meeting minutes are documents that record the content discussed and decisions made during a meeting.

[1057] "Emotion analysis" is the process of recognizing a user's emotional state based on their input and actions.

[1058] "Planning" refers to a concrete plan or concept for achieving a specific objective.

[1059] A "report" is a document that summarizes the progress and results of work or a project.

[1060] A "survey" is a method of collecting responses to questions set up for a specific research purpose.

[1061] "Employee training" refers to the provision of training and curricula designed to improve the skills and knowledge of employees and staff.

[1062] An "educational module" is a unit of learning content designed for a specific educational purpose.

[1063] "Video content" refers to the content of videos filmed for purposes such as promotion, explanation, or education.

[1064] A "shooting plan" is a plan that is formulated in advance to ensure efficient video shooting.

[1065] An "operation log" refers to a record of operations and events performed by a system or terminal.

[1066] A "call center" is a department or organization that functions as a point of contact for handling customer inquiries.

[1067] "Inquiry details" refers to the specific content of questions and consultations received by customers at the call center.

[1068] "Stress management" refers to methods for evaluating the stress that users and employees experience at work and appropriately mitigating it.

[1069] A "visual report" is a report that presents data in a visually easy-to-understand format, such as graphs and charts.

[1070] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[1071] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[1072] Hardware and software to be used

[1073] Emotion engine: Uses Microsoft Azure Cognitive Services or Google Cloud Emotion AI.

[1074] Servers: We will use AWS EC2 servers and Amazon RDS as the database service.

[1075] Devices: iOS / Android devices are accepted.

[1076] Video editing software: Adobe Premiere Pro will be used.

[1077] Log collection software: Use the ELK stack (Elasticsearch, Kibana, Logstash).

[1078] Emotional engine integration

[1079] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. This recognized emotion information is sent to the server and used in various business processes.

[1080] Daily schedule management

[1081] Users input their daily schedules and tasks from their respective devices, and their facial expressions and voices are analyzed by an emotion engine. The devices temporarily store the entered schedule information and send it to the server along with the emotion data. The server stores the schedule information and emotion data in a central database and synchronizes the latest schedule information and emotion feedback to each device.

[1082] Meeting minutes

[1083] During a meeting, the user enters meeting minutes into a terminal. The emotion engine recognizes the user's emotions while they are typing, detecting tension and stress. When the user clicks the save button after the meeting ends, the terminal sends the meeting minutes and emotion data to the server, which stores them in a central database.

[1084] Planning and report writing

[1085] Users create drafts of projects and reports using a terminal. The terminal provides users with necessary templates and past materials, and the emotion engine analyzes whether the user is experiencing stress or satisfaction. Completed projects and reports are sent to a server, stored in a central database, and notifications are sent to relevant parties. The emotion engine provides feedback tailored to the user's emotions.

[1086] Market research

[1087] The user sets market research questions on their device and sends the set questions to the server. The server generates a survey link and sends the link to each research participant based on the distribution list. The emotion engine analyzes the user's response during distribution. Research participants answer the survey via the link, and the data is sent from their device to the server. The server analyzes the collected response data and generates a visual report. The emotion engine uses this aggregated data to analyze emotional trends and reflects them in the results.

[1088] Creating a promotional video

[1089] The user uses a device to plan the video content and shooting schedule, and an emotion engine analyzes the user's emotions and incorporates them into the video content. The device then uses video editing software to edit the footage and sends the final version to the server. The server stores the completed video, generates the necessary links, and provides them to the user.

[1090] System testing and monitoring

[1091] The terminal executes a test script and sends the results to the server. The server collects operation logs from each terminal and monitors for anomalies. The sentiment engine evaluates the user's emotions while using the system and optimizes the user experience.

[1092] Call center initial support

[1093] Operators input customer inquiries into a terminal, and an emotion engine analyzes the operator's emotions during the interaction, providing stress reduction advice. The server stores the inquiry details in a central database and combines them with emotion data to evaluate the quality of customer service.

[1094] Examples of specific cases and prompt statements

[1095] For example, when a user uses the "market research" function, the process proceeds as follows:

[1096] 1. Users input market research questions from their devices, and an emotion engine analyzes the user's facial expressions and voice as they input the information.

[1097] 2. The terminal sends the entered questions to the server, and the server stores the questions and sentiment data in a central database.

[1098] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[1099] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[1100] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[1101] Example of a prompt:

[1102] "Based on the analysis of sentiment data from the following market research questions, please tell us what improvements can be made."

[1103] In this way, the system helps users conduct market research effectively and provides higher quality services by utilizing sentiment data.

[1104] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1105] Step 1: Enter schedule

[1106] 1.1 Users input their daily schedules and tasks from their respective devices. Input includes items such as "meetings," "document creation," and "lunch meetings."

[1107] 1.2 The terminal temporarily saves the entered schedule information. The input is the schedule information entered by the user.

[1108] 1.3 The emotion engine analyzes the user's facial expressions and voice during input and generates emotion data. Input data comes from the user's camera and microphone.

[1109] 1.4 The terminal sends schedule information and sentiment data to the server. The output includes schedule information and sentiment data.

[1110] 1.5 The server saves schedule information and sentiment data to the central database. Schedule information and sentiment data are saved to the central database as output.

[1111] Step 2: Schedule synchronization

[1112] 2.1 The server synchronizes the latest schedule information and sentiment feedback to each terminal. The input is information stored in a central database, and the output is updated data sent to the terminal.

[1113] 2.2 The device locally stores the schedule information and sentiment feedback it receives. A specific example is push notifications for a scheduling app.

[1114] Step 3: Enter meeting minutes

[1115] 3.1 Users enter meeting minutes into their terminals during the meeting. This input includes information such as "agenda," "decisions," and "next actions."

[1116] 3.2 The terminal temporarily saves meeting minutes in real time. The input is the meeting minutes information entered by the user, and the output is the temporarily saved data.

[1117] 3.3 The emotion engine recognizes the user's emotions during input and detects tension and stress. The user's facial expression data is the input, and emotion data is generated as the output.

[1118] Step 4: Send and save meeting minutes

[1119] 4.1 The user clicks the save button after the meeting ends. User actions are included as input.

[1120] 4.2 The terminal sends meeting minutes and sentiment data to the server. Meeting minutes and sentiment data are sent to the server as output.

[1121] 4.3 The server saves the received meeting minutes and sentiment data to the central database. The output includes the meeting minutes and sentiment data stored in the central database.

[1122] Step 5: Planning and Report Creation

[1123] 5.1 Users create drafts of plans and reports using their devices. The input includes the content of the plan.

[1124] 5.2 The terminal provides users with necessary templates and historical documents. Related documents are displayed to the user as output.

[1125] 5.3 The emotion engine analyzes whether the user is experiencing stress or satisfaction. Inputs include the user's keystroke patterns and mouse movements, and emotional data is generated as output.

[1126] 5.4 The terminal sends the completed project proposal or report to the server. The project proposal or report is sent to the server as output.

[1127] 5.5 The server stores them in a central database and sends notifications to the relevant parties. The output includes the stored data and the sent notifications.

[1128] 5.6 The emotion engine provides feedback that corresponds to the user's emotions. The feedback is displayed to the user as output.

[1129] Step 6: Market Research

[1130] 6.1 The user sets the market research questions on the device. The questions are included as input.

[1131] 6.2 The terminal sends the configured questions to the server. The question items are sent to the server as output.

[1132] 6.3 The server generates the survey link and sends the link to each survey participant based on the distribution list. The generated link and the link itself are sent as output.

[1133] 6.4 The emotion engine analyzes the user's reactions. User facial expression data is taken as input, and analyzed emotion data is generated as output.

[1134] 6.5 Participants answer the questionnaire via a link, and their devices send the data to the server. The response data is included as input.

[1135] 6.6 The server stores the collected response data, and the sentiment engine analyzes the sentiment trends. The output includes the stored response data and the analysis results.

[1136] Step 7: Create a promotional video

[1137] 7.1 The user uses a device to formulate the video content and shooting plan. The video content is included as input.

[1138] 7.2 The emotion engine analyzes the user's emotions and reflects them in the video content. User emotion data is the input, and it is reflected in the video content as the output.

[1139] 7.3 The terminal uses video editing software to edit the footage and sends the final version to the server. The edited video is sent to the server as output.

[1140] 7.4 The server stores the completed video and generates the necessary links to provide to the user. The links are generated and provided as output.

[1141] Step 8: System Testing and Monitoring

[1142] 8.1 The terminal executes the test script and sends the results to the server. The test results are included as input.

[1143] 8.2 The server collects operation logs from each terminal and monitors for anomalies. Operation logs are included as input.

[1144] 8.3 The emotion engine evaluates the user's emotions while using the system and optimizes the user experience. The input is user emotion data, and the output is optimized feedback.

[1145] Step 9: Temporary support at the call center

[1146] 9.1 An operator enters a customer inquiry into a terminal, and the emotion engine analyzes the response. The input includes the content of the inquiry.

[1147] 9.2 The terminal sends the entered query content to the server. The query data is sent to the server as output.

[1148] 9.3 The server stores the inquiry details in a central database and evaluates the quality of customer service by combining them with sentiment data. The output includes the stored data and the evaluation results.

[1149] (Application Example 2)

[1150] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1151] In modern brick-and-mortar stores, much effort is being made to improve customer satisfaction, and as part of this, it is necessary to accurately understand customer emotions and provide services based on those emotions. However, with previous systems, it was difficult to recognize customer emotions in real time and provide optimal feedback based on that information. Furthermore, it was difficult to centrally manage customer emotion data and use it to help with future visits. As a result, customer satisfaction tended to decline and the quality of service varied.

[1152] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and save it in a central database; means for users to input meeting minutes during a meeting and send them to the server after the meeting ends; means for users to create plans and reports, save them after completion, and notify relevant parties; means for users to set market research questions, distribute questionnaires, and collect and analyze response data; means for managing employee training curricula, distributing training modules, and reporting progress; means for users to formulate video content and shooting plans, edit them, and save the final version to the server; means for collecting system operation logs, detecting and reporting problems; means for inputting and recording inquiry content to call center operators; means for recognizing customer emotional information in real time and providing store staff with the most appropriate service according to that emotional state; and means for saving past visit data and emotional history and providing the most appropriate service on the next visit. This makes it possible to recognize customer emotional states in real time and provide the most appropriate service based on that information, thereby improving customer satisfaction and standardizing service quality.

[1153] A "user" refers to a person who uses a system to input, manipulate, and manage various types of information.

[1154] A "terminal" refers to a device used by a user to input and manipulate information. Examples include computers, smartphones, and tablets.

[1155] A "server" refers to a central system that receives, stores, and processes information sent by users.

[1156] A "central database" refers to a large-scale database where information managed by a system is stored centrally.

[1157] An "emotion engine" refers to a system that analyzes a user's facial expressions, voice, and behavioral patterns to recognize their emotions in real time.

[1158] "Schedule information" refers to information about appointments and tasks entered by the user.

[1159] "Meeting" refers to a conference or meeting where users enter meeting minutes.

[1160] "Meeting minutes" refers to a document that records the content discussed and decisions made during a meeting.

[1161] "Planning" refers to a plan or concept for achieving a specific objective.

[1162] A "report" refers to a document that summarizes the results of research and analysis on a specific theme or topic.

[1163] "Market research" refers to research activities conducted to understand consumer needs and market trends.

[1164] A "survey" refers to a method used in market research to distribute questions to target individuals and collect their responses.

[1165] "Employee training" refers to educational programs implemented with the aim of improving employees' skills and knowledge.

[1166] An "educational module" refers to an individual learning unit used for employee training.

[1167] "Video content" refers to information or plans that are recorded as video.

[1168] A "filming plan" refers to the plan and schedule for shooting the content of a video.

[1169] "Operation log" refers to the operation history and event records of a system or terminal.

[1170] A "call center" refers to a department that has a customer service function, handling inquiries and providing support.

[1171] An "operator" refers to a person who handles customer inquiries in a call center.

[1172] A "customer" refers to a person who visits a physical store and receives services.

[1173] "Visit data" refers to information about when a customer visits a store.

[1174] "Service" refers to the benefits and assistance that a store provides to its customers.

[1175] The following describes the specific mechanism and processing of the system as an embodiment of this invention. This system is realized through the collaboration of a user, a terminal, a server, and an emotion engine.

[1176] Program generation and processing flow

[1177] Hardware and software usage

[1178] Hardware: User devices include smartphones, tablets, smart glasses, and head-mounted displays. Terminals used by store staff, as well as cameras and sensors for analyzing emotional data, will also be used.

[1179] Software: Python, OpenCV, and an emotion recognition library (e.g., EmotionRecognition) are used. This allows for real-time recognition of customer facial expressions and voices by analyzing camera footage, and generates emotion data.

[1180] User

[1181] Users input various data such as schedule information, meeting minutes, plans, reports, and survey settings through their devices and send it to the server. In physical stores, emotional information of users when they visit as customers is also collected. For example, store staff using smart glasses scan customers' facial expressions and analyze that data.

[1182] terminal

[1183] The terminal temporarily stores user-entered data and sends it to the server. Furthermore, it analyzes emotional data collected through the emotion engine, monitors the results in real time, and provides appropriate feedback. In physical stores, it analyzes camera footage to understand customer emotions. It also manages past visit data and emotional history to provide optimal service for subsequent visits.

[1184] server

[1185] The server stores schedule information, meeting minutes, plans, reports, survey responses, training module progress, activity logs, and sentiment data submitted by users in a central database. It also synchronizes the latest information with each terminal and provides necessary notifications. In physical stores, it analyzes customer sentiment data in real time and supports the provision of optimal services based on the results.

[1186] Specific example

[1187] Consider a scenario where store staff at a physical store use smart glasses to analyze customers' emotions in real time. If the staff member detects that the customer is stressed, a notification will appear on their device saying, "Please provide a relaxing service." Furthermore, if there is data indicating that the customer has previously been relaxed, the system will suggest the most suitable service for that customer.

[1188] Example of a prompt

[1189] "We are developing an application that uses new technology to recognize customer emotions in real time and provide optimal feedback to store staff. Specifically, it's a system that analyzes camera footage to predict customer emotions and notifies staff of the results. Could you please provide a detailed code example for this system?"

[1190] Thus, this invention aims to improve customer service in physical stores by working in conjunction with an emotion engine.

[1191] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1192] Step 1:

[1193] The user enters schedule information into the terminal. The terminal temporarily stores the entered schedule information and sends it to the server. This schedule information includes the date, time, and details of the appointment. The entered data is temporarily stored on the terminal and sent to the server.

[1194] Step 2:

[1195] The server receives schedule information sent by users and stores it in a central database. The server analyzes the received schedule information and synchronizes the latest schedule information to each terminal. This ensures that all devices have the most up-to-date schedule information.

[1196] Step 3:

[1197] During a meeting, the user enters meeting minutes into a terminal. The terminal temporarily stores the entered minutes in real time and sends them to the server after the meeting ends. At this stage, the emotion engine analyzes the user's facial expressions and voice to generate emotion data. The meeting minutes and emotion data are temporarily stored on the terminal.

[1198] Step 4:

[1199] The server receives meeting minutes and sentiment data after meetings and stores them in a central database. The server then analyzes the stored data and generates necessary feedback. This process manages both the meeting minutes and sentiment data.

[1200] Step 5:

[1201] Users input plans and reports into a terminal, and upon completion, save them and notify relevant parties. The terminal temporarily stores drafts of the plans and reports, and sends them to the server upon completion. Even at this stage, the emotion engine monitors the user's state and analyzes stress and satisfaction levels.

[1202] Step 6:

[1203] The server receives the final versions of proposals and reports, along with sentiment data, and stores them in a central database. The server analyzes the received data and sends notifications to relevant parties. This facilitates the management of proposals and reports and the provision of sentiment feedback.

[1204] Step 7:

[1205] The user enters market research questions into a terminal and sends them to the server. When creating the survey, the emotion engine analyzes the user's state and generates emotion data. The server receives and stores the questions and emotion data.

[1206] Step 8:

[1207] The server generates a survey link and sends it to each participant based on the distribution list. During this sending process, the sentiment engine analyzes user responses. The collected survey response data is temporarily stored on the server.

[1208] Step 9:

[1209] The user enters their video shooting plan into the terminal and sends the completed video to the server. The terminal edits the video and temporarily saves the final version. During shooting, an emotion engine monitors the user's state and analyzes emotional data.

[1210] Step 10:

[1211] The server receives the completed video and sentiment data and stores them in a central database. The server generates the necessary links and provides them to the user. This allows for the management of video content and sentiment feedback.

[1212] Step 11:

[1213] Store staff use smart glasses to scan customers' facial expressions. The device collects customer facial data in real time, and an emotion engine analyzes it to generate emotion data. Real-time emotion information of the customer when they visit the store is displayed on the device.

[1214] Step 12:

[1215] The server receives customer sentiment data and stores it in a central database along with past visit data and sentiment history. Based on the analysis results, the server sends notifications to store staff to provide the best possible service. This improves the customer experience.

[1216] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1217] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1218] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1219] [Third Embodiment]

[1220] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1221] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1222] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1223] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1224] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1226] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1227] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1228] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1229] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1230] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1231] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1232] This invention relates to a system for centrally managing and efficiently executing various tasks. This system enables the efficient operation of a wide range of tasks through the collaboration of users, terminals, and servers.

[1233] Daily schedule management

[1234] 1. Schedule entry:

[1235] Users enter their daily schedules and tasks from each device.

[1236] The terminal temporarily saves the entered schedule and sends it to the server.

[1237] 2. Schedule saving and synchronization:

[1238] The server stores the received schedule information in a central database.

[1239] The server synchronizes the latest schedule information to each terminal after saving it.

[1240] Meeting minutes

[1241] 1. Entering meeting minutes:

[1242] The user enters meeting minutes into their terminal during the meeting.

[1243] The terminal temporarily saves meeting minutes in real time.

[1244] 2. Sending and saving meeting minutes:

[1245] The user clicks the save button after the meeting ends.

[1246] The terminal sends the meeting minutes to the server.

[1247] The server saves the received meeting minutes to a central database.

[1248] Planning and report writing

[1249] 1. Planning and input:

[1250] Users create drafts of plans and reports using their devices.

[1251] The terminal provides users with necessary templates and historical documents.

[1252] 2. Saving and Notifications:

[1253] The terminal sends completed plans and reports to the server.

[1254] The server stores them in a central database and sends notifications to the relevant parties.

[1255] Market research

[1256] 1. Creating and preparing the questionnaire:

[1257] The user sets the market research questions on their device.

[1258] The terminal sends the configured question to the server.

[1259] 2. Distribution of questionnaires:

[1260] The server generates a survey link and sends it to each recipient based on the distribution list.

[1261] 3. Response collection and analysis:

[1262] Participants will answer the questionnaire via the link they receive.

[1263] The device sends the response data to the server.

[1264] The server analyzes the collected response data, generates a visual report, and provides it to the user.

[1265] Employee training

[1266] 1. Curriculum management and delivery:

[1267] The server manages the employee training curriculum and distributes training modules to each terminal.

[1268] 2. Report on educational progress:

[1269] The terminal reports the progress of the educational module to the server in real time.

[1270] Users enter feedback after each session is completed and send it to the server.

[1271] Creating a promotional video

[1272] 1. Planning and developing video content:

[1273] The user uses the device to plan the video content and shooting schedule.

[1274] 2. Filming and editing:

[1275] The terminal uses video editing software to edit the footage and sends the final version to the server.

[1276] The server stores the completed video and generates the necessary links to provide to the user.

[1277] System testing and monitoring

[1278] 1. Run the test:

[1279] The terminal executes the test script and sends the results to the server.

[1280] 2. Log collection and monitoring:

[1281] The server collects operation logs from each terminal and monitors for anomalies.

[1282] The server will report any problems detected to the user.

[1283] Call center initial support

[1284] 1. Inquiry handling:

[1285] The operator enters the customer's inquiry into the terminal.

[1286] 2. Recording and Analysis:

[1287] The server stores the query content in a central database and performs analysis.

[1288] Specific example:

[1289] For example, when a user uses the "market research" function, the process proceeds as follows:

[1290] 1. The user sets the survey questions on their device and submits them.

[1291] 2. The server receives the questions and generates a survey link.

[1292] 3. The device sends the link to the person being surveyed.

[1293] 4. Participants will answer the questionnaire using the provided link.

[1294] 5. The device sends the response data to the server.

[1295] 6. The server aggregates the data, generates a report including the analysis results, and provides it to the user.

[1296] This allows the system to help users conduct market research effectively.

[1297] The following describes the processing flow.

[1298] Market research processing steps

[1299] Creating and preparing the questionnaire for distribution

[1300] Step 1:

[1301] The user enters market research survey questions from their device.

[1302] The terminal temporarily stores the entered questions in local storage and checks the integrity of the input.

[1303] Step 2:

[1304] The terminal sends the questions that have completed the integrity check to the server.

[1305] The server reviews the received questions and saves them to the central database.

[1306] Distribution of questionnaires

[1307] Step 3:

[1308] The server generates a link to the survey and emails the link to each survey participant based on the distribution list.

[1309] Step 4:

[1310] The terminal reports the transmission status to the server in real time.

[1311] The server confirms that the survey link has been successfully sent and updates the log.

[1312] Collection of responses

[1313] Step 5:

[1314] Participants click the received link to access the survey page.

[1315] The device displays the survey response screen, and the survey participants answer the questions.

[1316] Step 6:

[1317] The device temporarily stores the survey participants' responses in local storage and sends the data to the server once all responses have been completed.

[1318] Step 7:

[1319] The server saves the received response data to a central database and returns a confirmation message to the terminal acknowledging the successful saving.

[1320] Data aggregation and analysis

[1321] Step 8:

[1322] The server aggregates the stored response data and performs statistical analysis.

[1323] The server converts the analysis results into graphs and charts, generating visual reports.

[1324] Step 9:

[1325] The server sends the generated report to the terminal.

[1326] Users can view reports from their devices and provide feedback as needed.

[1327] Record of feedback

[1328] Step 10:

[1329] The device sends the user's input to the server.

[1330] The server saves the received feedback to a database and records it as an area for improvement in future market research.

[1331] The above processing steps enable the system to efficiently carry out the entire market research process, from data collection and analysis to reporting, in a consistent manner.

[1332] (Example 1)

[1333] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1334] Currently, there is a need for a centralized management system to efficiently handle multiple functions in business management and operations, such as schedule management, meeting minute creation, planning and report creation, market research, employee training, promotional video production, system testing and monitoring, and call center inquiry handling. When these tasks are managed individually, data synchronization and sharing become cumbersome, leading to decreased efficiency. Furthermore, there is a growing need for a system that reduces user effort while enabling accurate and rapid data processing.

[1335] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1336] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for terminals to temporarily store schedules and send them to the server, and means for the server to store schedule information in a central database and synchronize it with each terminal. This enables centralized management of schedule information and real-time data synchronization. It also includes means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to create plans and reports, save them after completion and notify relevant parties, and means for users to set market research questions, distribute questionnaires, and collect and analyze response data. The server also includes means for generating questionnaire links and sending them to each target based on a distribution list, means for managing employee training curricula, distributing training modules and reporting progress, means for users to formulate and edit video content and shooting plans and save the final version to the server, means for the server to collect system operation logs, detect and report problems, and means for call center operators to input and record inquiry details. This enables efficient and accurate centralized management of multiple tasks, significantly reducing user effort and improving overall operational efficiency.

[1337] A "user" refers to an entity that uses the system to perform various tasks.

[1338] A "terminal" refers to a device operated by a user, such as a computer or smartphone used for tasks like entering schedules, creating meeting minutes, or designing surveys.

[1339] A "server" refers to a central computer system used to store data and manage communication and data synchronization between terminals.

[1340] A "central database" refers to a database built on a server that centrally manages and stores all data generated within the system.

[1341] "Schedule information" refers to data about appointments and tasks entered by the user.

[1342] "Meeting minutes" refers to a document that users record during a meeting, including details of the meeting, discussions, and decisions made.

[1343] "Plans" and "reports" refer to plans and reports created by users, which are documents containing information about the business or project.

[1344] "Market research" refers to the process of collecting market reactions and opinions through surveys set by users.

[1345] A "survey link" is a URL generated by the server and sent to the survey participant, used to access the survey form.

[1346] An "employee training curriculum" refers to an educational program aimed at improving employees' skills and acquiring knowledge.

[1347] An "education module" is a part of the employee training curriculum and refers to the specific content of each lesson or session.

[1348] "Video content" refers to the scenario and structure that the user develops when creating a promotional video.

[1349] A "shooting plan" refers to a specific plan for shooting a video, including the shooting schedule, locations, and necessary equipment.

[1350] "Operation log" refers to data that records the operating status of a system or terminal.

[1351] A "call center operator" refers to a person who handles customer inquiries at a call center.

[1352] This invention relates to a system for centrally managing and efficiently executing various tasks. Through collaboration between users, terminals, and servers, it enables the efficient operation of a wide range of tasks.

[1353] Hardware and software to be used

[1354] hardware

[1355] Devices: Laptops, smartphones (iOS / Android), tablets (iPad), desktop PCs

[1356] Server: Cloud servers (AWS, Azure, Google Cloud)

[1357] software

[1358] Database management systems (PostgreSQL, MySQL)

[1359] Text editor

[1360] Video editing software

[1361] Communication method (Firebase Cloud Messaging, HTTPS)

[1362] Email sending system (SendGrid)

[1363] Analysis tools (Python Pandas, Matplotlib)

[1364] The specific embodiments of each function of this system are described below.

[1365] Schedule management

[1366] Input and data transmission

[1367] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[1368] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[1369] Data storage and synchronization

[1370] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[1371] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[1372] Market research

[1373] Questionnaire creation and distribution

[1374] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[1375] The terminal POSTs the question data to the server in JSON format.

[1376] Generate and send survey link

[1377] The server uses a URL shortening service to generate the survey link.

[1378] The server will send this link to the recipients using the email sending system (SendGrid).

[1379] Response collection and analysis

[1380] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[1381] The device temporarily stores the response data and sends it to the server in JSON format.

[1382] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[1383] Example of a prompt

[1384] Schedule management:

[1385] "Please add a 10:00 meeting to this week's schedule and save it to the server."

[1386] Market research:

[1387] "Please create a questionnaire about the new product and send it to the target audience."

[1388] The various functions of this system allow users to efficiently centralize and manage multiple tasks, significantly reducing effort and improving overall operational efficiency.

[1389] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1390] Daily schedule management

[1391] Step 1:

[1392] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[1393] Input: Date, time, meeting content (e.g., 10:00, meeting with a client)

[1394] Action: The user creates a new schedule in the calendar app and clicks the save button.

[1395] Output: Schedule data stored in the local cache

[1396] Step 2:

[1397] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[1398] Input: Schedule data (e.g., 10:00, Meeting with a client)

[1399] Operation: Save to local cache and convert to JSON format.

[1400] Output: POST request to the server

[1401] Step 3:

[1402] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[1403] Input: Schedule data in JSON format

[1404] Operation: Data analysis and insertion into a PostgreSQL database.

[1405] Output: Schedule data stored in the database

[1406] Step 4:

[1407] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[1408] Input: Schedule data stored in the database

[1409] Operation: Triggering push notifications using Firebase Cloud Messaging

[1410] Output: Latest schedule notification received on each device

[1411] Market research

[1412] Step 1:

[1413] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[1414] Input: Question item (e.g., What do you think of the product?)

[1415] Operation: Enter and save a new question on the web interface.

[1416] Output: Question data stored in the local cache

[1417] Step 2:

[1418] The terminal POSTs the question data to the server in JSON format.

[1419] Input: Question data (e.g., What do you think of the product?)

[1420] Operation: Save to local cache and convert to JSON format.

[1421] Output: POST request to the server

[1422] Step 3:

[1423] The server uses a URL shortening service to generate the survey link.

[1424] Input: Question data

[1425] Operation: Generates survey links using a URL shortening service.

[1426] Output: Generated survey link

[1427] Step 4:

[1428] The server will send this link to the recipients using the email sending system (SendGrid).

[1429] Input: Generated survey link

[1430] Operation: Send emails based on distribution lists using SendGrid.

[1431] Output: Email and survey link sent to participants

[1432] Step 5:

[1433] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[1434] Input: Response data entered by survey participants.

[1435] How it works: Fill out the survey form in your web browser and submit your response.

[1436] Output: Sending response data to the server

[1437] Step 6:

[1438] The device temporarily stores the response data and sends it to the server in JSON format.

[1439] Input: Response data

[1440] Operation: Save to local cache and convert to JSON format.

[1441] Output: POST request to the server

[1442] Step 7:

[1443] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[1444] Input: Response data in JSON format

[1445] Function: Saves to database, runs analytical algorithms, generates visual reports (Pandas, Matplotlib)

[1446] Output: Visual reports displayed on the user's dashboard.

[1447] Example of a prompt

[1448] Schedule Management: "Add a 10:00 meeting to this week's schedule and save it to the server."

[1449] Market research: "Create a questionnaire about the new product and send it to the target audience."

[1450] This program's processing steps allow users to efficiently manage schedules and conduct market research, enabling them to manage various tasks in an integrated manner.

[1451] (Application Example 1)

[1452] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1453] Traditional business management systems have struggled to efficiently integrate scheduling, real-time tracking of operations, meeting minute creation, employee training, and system testing and monitoring within logistics centers. In particular, the inability to track the movement of goods and the progress of picking lists within the logistics center in real time led to decreased operational efficiency and increased errors. A new system is needed to address these challenges.

[1454] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1455] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to set market research questions, distribute questionnaires, and collect and analyze response data, and means for tracking the movement of goods within the logistics center and the progress of picking lists in real time. This enables efficient operation of the entire logistics center and real-time management of various operations.

[1456] A "user" is an entity that uses a system to perform various tasks.

[1457] A "terminal" is a device used by a user to input information or utilize system functions.

[1458] A "central database" is a storage device used to centrally store information for the entire system.

[1459] A "server" is a computer system that receives and processes data transmitted from various terminals.

[1460] "Schedule information" refers to the daily schedules and tasks entered by the user.

[1461] "Meeting minutes" are documents that users record during a meeting and save afterward.

[1462] A "project" is a new project or plan created by a user.

[1463] A "report" is a document that summarizes the progress and results of a project.

[1464] "Market research" is the activity of collecting and analyzing data about a specific market.

[1465] A "questionnaire" is a survey conducted as part of market research, consisting of a series of questions.

[1466] "Data collection" refers to the act of gathering information transmitted from each device.

[1467] "Data analysis" is the process of verifying collected information and compiling it into a visual report.

[1468] An "employee training curriculum" is a learning program aimed at improving employees' skills.

[1469] An "education module" refers to software and teaching materials used to implement employee training curricula.

[1470] "Feedback" refers to the evaluations and opinions that users enter after completing an educational module.

[1471] A "promotional video" is video content produced for the purpose of advertising or public relations.

[1472] "Real-time tracking" is a function that allows you to instantly grasp the status of ongoing operations and logistics.

[1473] A "picking list" is a list that displays the items needed for a specific order or task.

[1474] This invention is a system for efficiently managing and operating operations at a logistics center. The specific program and its processing details for realizing this system are described below.

[1475] System Configuration

[1476] This system consists primarily of users, production terminals, an information processing server, and a central database. The terminals provide an interface for users to input information and send it to the server. The server stores the received information in the central database and synchronizes it with other terminals as needed.

[1477] Schedule management

[1478] Users input their daily tasks and schedules within the logistics center using terminals. This schedule information is temporarily stored on the terminals and then sent to the server. The server stores the received schedule information in a central database and synchronizes the latest schedule information with each terminal.

[1479] Real-time tracking

[1480] Data is entered via terminals to track the movement of goods within the logistics center and the progress of picking lists in real time. The terminals transmit this information to a server in real time, and the server stores it in a central database, thereby improving the efficiency of goods management.

[1481] Meeting minutes

[1482] During the meeting, users enter meeting minutes into their terminals. After the meeting ends, users click the save button, and the minutes are sent from their terminals to the server. The server saves the received minutes to a central database and sends notifications to the relevant parties.

[1483] Employee training

[1484] The server manages the employee training curriculum and delivers training modules to each terminal. Training progress reports are sent from the terminal to the server in real time. Users enter feedback after completing each session and send it to the server.

[1485] System testing and monitoring

[1486] The terminal collects system operation logs and sends them to the server. The server stores the operation logs of each terminal in a central database and reports any abnormalities to the user.

[1487] Specific example

[1488] For example, to add a new inventory check task, the user would enter the following information on the schedule input screen:

[1489] Example of a schedule input prompt

[1490] Please add a new task.

[1491] User: employee1

[1492] Task name: Inventory check

[1493] Date: 2023-10-10

[1494] Time: 14:00

[1495] In this way, the system enables efficient operation of the entire logistics center and real-time management of various tasks. Furthermore, it can streamline other incidental tasks such as schedule management and meeting minute creation.

[1496] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1497] Step 1:

[1498] The user opens the schedule entry screen using their device and enters their daily tasks and appointments. The entered data includes the username, task name, date, and time. The device temporarily saves the entered data and then sends it to the server.

[1499] Step 2:

[1500] The server receives schedule information sent from the terminals. The data includes the username, task name, date, and time, and this data is stored in a central database. After the schedule information is stored, the server sends a request to each terminal to synchronize with the latest schedule information.

[1501] Step 3:

[1502] Users use terminals to input information about the movement of items within the logistics center and picking lists in real time. The entered data is immediately sent from the terminal to the server.

[1503] Step 4:

[1504] The server receives real-time tracking data sent from the terminal and stores it in a central database. Based on the received data, the server monitors the movement of items and the progress of picking lists in real time.

[1505] Step 5:

[1506] During a meeting, the user enters meeting minutes using their device. After the meeting ends, the user clicks the save button on their device. The device temporarily saves the meeting minutes and sends them to the server.

[1507] Step 6:

[1508] The server receives meeting minutes data sent from terminals and stores it in a central database. This data includes the meeting title, agenda, participants, and minutes content. The server notifies relevant parties that the minutes have been saved.

[1509] Step 7:

[1510] Users access the employee training curriculum from their devices and utilize the training modules. After completing each session, users provide feedback. The device then sends the feedback and progress report to the server.

[1511] Step 8:

[1512] The server receives progress data and feedback on educational modules sent from terminals and stores them in a central database. The server manages educational progress and feedback in real time.

[1513] Step 9:

[1514] The terminal collects system operation status as logs and periodically sends them to the server. The log data includes operational anomalies, error messages, and performance metrics.

[1515] Step 10:

[1516] The server receives operation logs sent from the terminal and stores them in a central database. The server analyzes the log data and issues a warning to the user if an anomaly is detected.

[1517] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1518] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[1519] Emotional engine integration

[1520] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[1521] Daily schedule management

[1522] 1. Schedule entry:

[1523] Users enter their daily schedules and tasks from each device.

[1524] The terminal temporarily stores the entered schedule information and sends it to the server.

[1525] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data, which is then sent to the server.

[1526] 2. Schedule saving and synchronization:

[1527] The server stores schedule information and sentiment data in a central database.

[1528] The server synchronizes the latest schedule information and emotional feedback to each terminal.

[1529] Meeting minutes

[1530] 1. Entering meeting minutes:

[1531] The user enters meeting minutes into their terminal during the meeting.

[1532] The terminal temporarily saves meeting minutes in real time.

[1533] The emotion engine recognizes the user's emotions while they are typing and detects tension and stress.

[1534] 2. Sending and saving meeting minutes:

[1535] The user clicks the save button after the meeting ends.

[1536] The terminal sends meeting minutes and sentiment data to the server.

[1537] The server stores the received meeting minutes and sentiment data in a central database.

[1538] Planning and report writing

[1539] 1. Planning and input:

[1540] Users create drafts of plans and reports using their devices.

[1541] The terminal provides users with necessary templates and historical documents.

[1542] The emotion engine analyzes whether the user is experiencing stress or satisfaction.

[1543] 2. Saving and Notifications:

[1544] The terminal sends completed plans and reports to the server.

[1545] The server stores them in a central database and sends notifications to the relevant parties.

[1546] The emotion engine provides feedback that responds to the user's emotions.

[1547] Market research

[1548] 1. Creating and preparing the questionnaire:

[1549] The user sets the market research questions on their device.

[1550] The terminal sends the configured question to the server.

[1551] 2. Distribution of questionnaires:

[1552] The server generates the survey link and sends the link to each survey participant based on the distribution list.

[1553] The emotion engine analyzes the user's reaction when a link is sent.

[1554] 3. Response collection and analysis:

[1555] Participants click the received link to access the survey page.

[1556] The device displays the survey response screen, and the survey participants answer the questions.

[1557] The device sends the response data to the server.

[1558] The server analyzes the collected response data and generates a visual report.

[1559] The emotion engine analyzes emotional tendencies based on aggregated data and reflects them in the results.

[1560] Employee training

[1561] 1. Curriculum management and delivery:

[1562] The server manages the employee training curriculum and distributes training modules to each terminal.

[1563] The emotion engine adjusts the curriculum based on employee emotional data.

[1564] 2. Report on educational progress:

[1565] The terminal reports the progress of the educational module to the server in real time.

[1566] Users enter feedback after each session is completed and send it to the server.

[1567] The server receives feedback and sentiment data and analyzes the educational effect.

[1568] Creating a promotional video

[1569] 1. Planning and developing video content:

[1570] The user uses the device to plan the video content and shooting schedule.

[1571] The emotion engine analyzes the user's emotions and reflects them in the video content.

[1572] 2. Filming and editing:

[1573] The terminal uses video editing software to edit the footage and sends the final version to the server.

[1574] The server stores the completed video and generates the necessary links to provide to the user.

[1575] System testing and monitoring

[1576] 1. Run the test:

[1577] The terminal executes the test script and sends the results to the server.

[1578] 2. Log collection and monitoring:

[1579] The server collects operation logs from each terminal and monitors for anomalies.

[1580] The emotion engine evaluates the user's emotions while using the system and optimizes the user experience.

[1581] Call center initial support

[1582] 1. Inquiry handling:

[1583] The operator enters the customer's inquiry into the terminal.

[1584] The emotion engine analyzes the operator's emotions during interactions and provides advice for stress reduction.

[1585] 2. Recording and Analysis:

[1586] The server stores the query content in a central database and performs analysis.

[1587] The emotion engine combines collected data with emotional data to evaluate the quality of customer service.

[1588] Specific example:

[1589] For example, when a user uses the "market research" function, the process proceeds as follows:

[1590] 1. The user inputs market research questions from their device. The emotion engine analyzes the user's facial expressions and voice as they input the information.

[1591] 2. The terminal sends the entered questions to the server. The server stores the questions and sentiment data in a central database.

[1592] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[1593] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[1594] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[1595] This allows the system to help users conduct market research effectively and to provide higher quality services by utilizing sentiment data.

[1596] The following describes the processing flow.

[1597] Processing steps in market research

[1598] Creating and preparing the questionnaire for distribution

[1599] Step 1:

[1600] The user enters market research questions from their terminal.

[1601] The device temporarily stores the entered questions and sends them to the emotion engine.

[1602] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data.

[1603] The emotion engine sends emotional data back to the device.

[1604] Step 2:

[1605] The device sends the questionnaire items and sentiment data to the server.

[1606] The server stores the questionnaire items and sentiment data in a central database.

[1607] Distribution of questionnaires

[1608] Step 3:

[1609] The server generates a survey link and sends the link via email to each survey participant based on the distribution list.

[1610] Step 4:

[1611] The terminal reports the transmission status to the server in real time.

[1612] The server logs the successful submission of the survey link.

[1613] The emotion engine re-analyzes the user's response during this process and updates the emotion data.

[1614] Collection of responses

[1615] Step 5:

[1616] Participants in the survey will access the questionnaire page by clicking the link they receive.

[1617] The device displays the survey response screen, and the survey participants answer the questions.

[1618] Step 6:

[1619] The device temporarily stores the survey participants' responses and sends the data to the server once all responses have been completed.

[1620] Step 7:

[1621] The server stores the response data in a central database.

[1622] The emotion engine analyzes the emotional data of the survey participants on the server and stores it together with the data in the database.

[1623] The server sends a confirmation message to the terminal acknowledging that the save was successful.

[1624] Data aggregation and analysis

[1625] Step 8:

[1626] The server aggregates the stored response data and performs statistical analysis.

[1627] The emotion engine works with the server to analyze emotional tendencies based on aggregated data and reflect them in the results.

[1628] Step 9:

[1629] The server converts the analysis results into graphs and charts, generating visual reports.

[1630] The server sends the generated report and sentiment analysis results to the terminal.

[1631] Step 10:

[1632] Users can view reports from their devices and provide feedback as needed.

[1633] The device sends user feedback to the emotion engine.

[1634] The emotion engine analyzes feedback and emotional data to derive areas for improvement for the next market research.

[1635] The emotion engine sends the analysis results to the server and stores them in the database.

[1636] This allows the system to efficiently manage the entire market research process and leverage user sentiment data to provide high-quality feedback and analytical results.

[1637] (Example 2)

[1638] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1639] Traditional business management systems focus on individual tasks and schedule management, but lack mechanisms to provide feedback that takes into account the user's emotional state. Furthermore, few systems utilize emotional data to improve work efficiency and quality. As a result, there is no effective method for managing user stress and satisfaction, making it a challenge to improve work quality and employee mental health.

[1640] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database along with emotional data; means for users to input meeting minutes during a meeting, perform emotional analysis, and send them to the server after the meeting ends; and means for users to create plans and reports, receive emotional feedback, and then save them and notify relevant parties after completion. This enables the analysis of users' emotional data in real time and provides optimal feedback based on that analysis, thereby improving work efficiency and the mental health of employees.

[1641] "Schedule information" refers to appointments and action plans that users enter to manage their daily work and tasks.

[1642] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions, voice, behavioral patterns, and other factors.

[1643] A "central database" is a database used to centrally manage and store data for the entire system.

[1644] Meeting minutes are documents that record the content discussed and decisions made during a meeting.

[1645] "Emotion analysis" is the process of recognizing a user's emotional state based on their input and actions.

[1646] "Planning" refers to a concrete plan or concept for achieving a specific objective.

[1647] A "report" is a document that summarizes the progress and results of work or a project.

[1648] A "survey" is a method of collecting responses to questions set up for a specific research purpose.

[1649] "Employee training" refers to the provision of training and curricula designed to improve the skills and knowledge of employees and staff.

[1650] An "educational module" is a unit of learning content designed for a specific educational purpose.

[1651] "Video content" refers to the content of videos filmed for purposes such as promotion, explanation, or education.

[1652] A "shooting plan" is a plan that is formulated in advance to ensure efficient video shooting.

[1653] An "operation log" refers to a record of operations and events performed by a system or terminal.

[1654] A "call center" is a department or organization that functions as a point of contact for handling customer inquiries.

[1655] "Inquiry details" refers to the specific content of questions and consultations received by customers at the call center.

[1656] "Stress management" refers to methods for evaluating the stress that users and employees experience at work and appropriately mitigating it.

[1657] A "visual report" is a report that presents data in a visually easy-to-understand format, such as graphs and charts.

[1658] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[1659] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[1660] Hardware and software to be used

[1661] Emotion engine: Uses Microsoft Azure Cognitive Services or Google Cloud Emotion AI.

[1662] Servers: We will use AWS EC2 servers and Amazon RDS as the database service.

[1663] Devices: iOS / Android devices are accepted.

[1664] Video editing software: Adobe Premiere Pro will be used.

[1665] Log collection software: Use the ELK stack (Elasticsearch, Kibana, Logstash).

[1666] Emotional engine integration

[1667] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. This recognized emotion information is sent to the server and used in various business processes.

[1668] Daily schedule management

[1669] Users input their daily schedules and tasks from their respective devices, and their facial expressions and voices are analyzed by an emotion engine. The devices temporarily store the entered schedule information and send it to the server along with the emotion data. The server stores the schedule information and emotion data in a central database and synchronizes the latest schedule information and emotion feedback to each device.

[1670] Meeting minutes

[1671] During a meeting, the user enters meeting minutes into a terminal. The emotion engine recognizes the user's emotions while they are typing, detecting tension and stress. When the user clicks the save button after the meeting ends, the terminal sends the meeting minutes and emotion data to the server, which stores them in a central database.

[1672] Planning and report writing

[1673] Users create drafts of projects and reports using a terminal. The terminal provides users with necessary templates and past materials, and the emotion engine analyzes whether the user is experiencing stress or satisfaction. Completed projects and reports are sent to a server, stored in a central database, and notifications are sent to relevant parties. The emotion engine provides feedback tailored to the user's emotions.

[1674] Market research

[1675] The user sets market research questions on their device and sends the set questions to the server. The server generates a survey link and sends the link to each research participant based on the distribution list. The emotion engine analyzes the user's response during distribution. Research participants answer the survey via the link, and the data is sent from their device to the server. The server analyzes the collected response data and generates a visual report. The emotion engine uses this aggregated data to analyze emotional trends and reflects them in the results.

[1676] Creating a promotional video

[1677] The user uses a device to plan the video content and shooting schedule, and an emotion engine analyzes the user's emotions and incorporates them into the video content. The device then uses video editing software to edit the footage and sends the final version to the server. The server stores the completed video, generates the necessary links, and provides them to the user.

[1678] System testing and monitoring

[1679] The terminal executes a test script and sends the results to the server. The server collects operation logs from each terminal and monitors for anomalies. The sentiment engine evaluates the user's emotions while using the system and optimizes the user experience.

[1680] Call center initial support

[1681] Operators input customer inquiries into a terminal, and an emotion engine analyzes the operator's emotions during the interaction, providing stress reduction advice. The server stores the inquiry details in a central database and combines them with emotion data to evaluate the quality of customer service.

[1682] Examples of specific cases and prompt statements

[1683] For example, when a user uses the "market research" function, the process proceeds as follows:

[1684] 1. Users input market research questions from their devices, and an emotion engine analyzes the user's facial expressions and voice as they input the information.

[1685] 2. The terminal sends the entered questions to the server, and the server stores the questions and sentiment data in a central database.

[1686] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[1687] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[1688] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[1689] Example of a prompt:

[1690] "Based on the analysis of sentiment data from the following market research questions, please tell us what improvements can be made."

[1691] In this way, the system helps users conduct market research effectively and provides higher quality services by utilizing sentiment data.

[1692] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1693] Step 1: Enter schedule

[1694] 1.1 Users input their daily schedules and tasks from their respective devices. Input includes items such as "meetings," "document creation," and "lunch meetings."

[1695] 1.2 The terminal temporarily saves the entered schedule information. The input is the schedule information entered by the user.

[1696] 1.3 The emotion engine analyzes the user's facial expressions and voice during input and generates emotion data. Input data comes from the user's camera and microphone.

[1697] 1.4 The terminal sends schedule information and sentiment data to the server. The output includes schedule information and sentiment data.

[1698] 1.5 The server saves schedule information and sentiment data to the central database. Schedule information and sentiment data are saved to the central database as output.

[1699] Step 2: Schedule synchronization

[1700] 2.1 The server synchronizes the latest schedule information and sentiment feedback to each terminal. The input is information stored in a central database, and the output is updated data sent to the terminal.

[1701] 2.2 The device locally stores the schedule information and sentiment feedback it receives. A specific example is push notifications for a scheduling app.

[1702] Step 3: Enter meeting minutes

[1703] 3.1 Users enter meeting minutes into their terminals during the meeting. This input includes information such as "agenda," "decisions," and "next actions."

[1704] 3.2 The terminal temporarily saves meeting minutes in real time. The input is the meeting minutes information entered by the user, and the output is the temporarily saved data.

[1705] 3.3 The emotion engine recognizes the user's emotions during input and detects tension and stress. The user's facial expression data is the input, and emotion data is generated as the output.

[1706] Step 4: Send and save meeting minutes

[1707] 4.1 The user clicks the save button after the meeting ends. User actions are included as input.

[1708] 4.2 The terminal sends meeting minutes and sentiment data to the server. Meeting minutes and sentiment data are sent to the server as output.

[1709] 4.3 The server saves the received meeting minutes and sentiment data to the central database. The output includes the meeting minutes and sentiment data stored in the central database.

[1710] Step 5: Planning and Report Creation

[1711] 5.1 Users create drafts of plans and reports using their devices. The input includes the content of the plan.

[1712] 5.2 The terminal provides users with necessary templates and historical documents. Related documents are displayed to the user as output.

[1713] 5.3 The emotion engine analyzes whether the user is experiencing stress or satisfaction. Inputs include the user's keystroke patterns and mouse movements, and emotional data is generated as output.

[1714] 5.4 The terminal sends the completed project proposal or report to the server. The project proposal or report is sent to the server as output.

[1715] 5.5 The server stores them in a central database and sends notifications to the relevant parties. The output includes the stored data and the sent notifications.

[1716] 5.6 The emotion engine provides feedback that corresponds to the user's emotions. The feedback is displayed to the user as output.

[1717] Step 6: Market Research

[1718] 6.1 The user sets the market research questions on the device. The questions are included as input.

[1719] 6.2 The terminal sends the configured questions to the server. The question items are sent to the server as output.

[1720] 6.3 The server generates the survey link and sends the link to each survey participant based on the distribution list. The generated link and the link itself are sent as output.

[1721] 6.4 The emotion engine analyzes the user's reactions. User facial expression data is taken as input, and analyzed emotion data is generated as output.

[1722] 6.5 Participants answer the questionnaire via a link, and their devices send the data to the server. The response data is included as input.

[1723] 6.6 The server stores the collected response data, and the sentiment engine analyzes the sentiment trends. The output includes the stored response data and the analysis results.

[1724] Step 7: Create a promotional video

[1725] 7.1 The user uses a device to formulate the video content and shooting plan. The video content is included as input.

[1726] 7.2 The emotion engine analyzes the user's emotions and reflects them in the video content. User emotion data is the input, and it is reflected in the video content as the output.

[1727] 7.3 The terminal uses video editing software to edit the footage and sends the final version to the server. The edited video is sent to the server as output.

[1728] 7.4 The server stores the completed video and generates the necessary links to provide to the user. The links are generated and provided as output.

[1729] Step 8: System Testing and Monitoring

[1730] 8.1 The terminal executes the test script and sends the results to the server. The test results are included as input.

[1731] 8.2 The server collects operation logs from each terminal and monitors for anomalies. Operation logs are included as input.

[1732] 8.3 The emotion engine evaluates the user's emotions while using the system and optimizes the user experience. The input is user emotion data, and the output is optimized feedback.

[1733] Step 9: Temporary support at the call center

[1734] 9.1 An operator enters a customer inquiry into a terminal, and the emotion engine analyzes the response. The input includes the content of the inquiry.

[1735] 9.2 The terminal sends the entered query content to the server. The query data is sent to the server as output.

[1736] 9.3 The server stores the inquiry details in a central database and evaluates the quality of customer service by combining them with sentiment data. The output includes the stored data and the evaluation results.

[1737] (Application Example 2)

[1738] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1739] In modern brick-and-mortar stores, much effort is being made to improve customer satisfaction, and as part of this, it is necessary to accurately understand customer emotions and provide services based on those emotions. However, with previous systems, it was difficult to recognize customer emotions in real time and provide optimal feedback based on that information. Furthermore, it was difficult to centrally manage customer emotion data and use it to help with future visits. As a result, customer satisfaction tended to decline and the quality of service varied.

[1740] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and save it in a central database; means for users to input meeting minutes during a meeting and send them to the server after the meeting ends; means for users to create plans and reports, save them after completion, and notify relevant parties; means for users to set market research questions, distribute questionnaires, and collect and analyze response data; means for managing employee training curricula, distributing training modules, and reporting progress; means for users to formulate video content and shooting plans, edit them, and save the final version to the server; means for collecting system operation logs, detecting and reporting problems; means for inputting and recording inquiry content to call center operators; means for recognizing customer emotional information in real time and providing store staff with the most appropriate service according to that emotional state; and means for saving past visit data and emotional history and providing the most appropriate service on the next visit. This makes it possible to recognize customer emotional states in real time and provide the most appropriate service based on that information, thereby improving customer satisfaction and standardizing service quality.

[1741] A "user" refers to a person who uses a system to input, manipulate, and manage various types of information.

[1742] A "terminal" refers to a device used by a user to input and manipulate information. Examples include computers, smartphones, and tablets.

[1743] A "server" refers to a central system that receives, stores, and processes information sent by users.

[1744] A "central database" refers to a large-scale database where information managed by a system is stored centrally.

[1745] An "emotion engine" refers to a system that analyzes a user's facial expressions, voice, and behavioral patterns to recognize their emotions in real time.

[1746] "Schedule information" refers to information about appointments and tasks entered by the user.

[1747] "Meeting" refers to a conference or meeting where users enter meeting minutes.

[1748] "Meeting minutes" refers to a document that records the content discussed and decisions made during a meeting.

[1749] "Planning" refers to a plan or concept for achieving a specific objective.

[1750] A "report" refers to a document that summarizes the results of research and analysis on a specific theme or topic.

[1751] "Market research" refers to research activities conducted to understand consumer needs and market trends.

[1752] A "survey" refers to a method used in market research to distribute questions to target individuals and collect their responses.

[1753] "Employee training" refers to educational programs implemented with the aim of improving employees' skills and knowledge.

[1754] An "educational module" refers to an individual learning unit used for employee training.

[1755] "Video content" refers to information or plans that are recorded as video.

[1756] A "filming plan" refers to the plan and schedule for shooting the content of a video.

[1757] "Operation log" refers to the operation history and event records of a system or terminal.

[1758] A "call center" refers to a department that has a customer service function, handling inquiries and providing support.

[1759] An "operator" refers to a person who handles customer inquiries in a call center.

[1760] A "customer" refers to a person who visits a physical store and receives services.

[1761] "Visit data" refers to information about when a customer visits a store.

[1762] "Service" refers to the benefits and assistance that a store provides to its customers.

[1763] The following describes the specific mechanism and processing of the system as an embodiment of this invention. This system is realized through the collaboration of a user, a terminal, a server, and an emotion engine.

[1764] Program generation and processing flow

[1765] Hardware and software usage

[1766] Hardware: User devices include smartphones, tablets, smart glasses, and head-mounted displays. Terminals used by store staff, as well as cameras and sensors for analyzing emotional data, will also be used.

[1767] Software: Python, OpenCV, and an emotion recognition library (e.g., EmotionRecognition) are used. This allows for real-time recognition of customer facial expressions and voices by analyzing camera footage, and generates emotion data.

[1768] User

[1769] Users input various data such as schedule information, meeting minutes, plans, reports, and survey settings through their devices and send it to the server. In physical stores, emotional information of users when they visit as customers is also collected. For example, store staff using smart glasses scan customers' facial expressions and analyze that data.

[1770] terminal

[1771] The terminal temporarily stores user-entered data and sends it to the server. Furthermore, it analyzes emotional data collected through the emotion engine, monitors the results in real time, and provides appropriate feedback. In physical stores, it analyzes camera footage to understand customer emotions. It also manages past visit data and emotional history to provide optimal service for subsequent visits.

[1772] server

[1773] The server stores schedule information, meeting minutes, plans, reports, survey responses, training module progress, activity logs, and sentiment data submitted by users in a central database. It also synchronizes the latest information with each terminal and provides necessary notifications. In physical stores, it analyzes customer sentiment data in real time and supports the provision of optimal services based on the results.

[1774] Specific example

[1775] Consider a scenario where store staff at a physical store use smart glasses to analyze customers' emotions in real time. If the staff member detects that the customer is stressed, a notification will appear on their device saying, "Please provide a relaxing service." Furthermore, if there is data indicating that the customer has previously been relaxed, the system will suggest the most suitable service for that customer.

[1776] Example of a prompt

[1777] "We are developing an application that uses new technology to recognize customer emotions in real time and provide optimal feedback to store staff. Specifically, it's a system that analyzes camera footage to predict customer emotions and notifies staff of the results. Could you please provide a detailed code example for this system?"

[1778] Thus, this invention aims to improve customer service in physical stores by working in conjunction with an emotion engine.

[1779] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1780] Step 1:

[1781] The user enters schedule information into the terminal. The terminal temporarily stores the entered schedule information and sends it to the server. This schedule information includes the date, time, and details of the appointment. The entered data is temporarily stored on the terminal and sent to the server.

[1782] Step 2:

[1783] The server receives schedule information sent by users and stores it in a central database. The server analyzes the received schedule information and synchronizes the latest schedule information to each terminal. This ensures that all devices have the most up-to-date schedule information.

[1784] Step 3:

[1785] During a meeting, the user enters meeting minutes into a terminal. The terminal temporarily stores the entered minutes in real time and sends them to the server after the meeting ends. At this stage, the emotion engine analyzes the user's facial expressions and voice to generate emotion data. The meeting minutes and emotion data are temporarily stored on the terminal.

[1786] Step 4:

[1787] The server receives meeting minutes and sentiment data after meetings and stores them in a central database. The server then analyzes the stored data and generates necessary feedback. This process manages both the meeting minutes and sentiment data.

[1788] Step 5:

[1789] Users input plans and reports into a terminal, and upon completion, save them and notify relevant parties. The terminal temporarily stores drafts of the plans and reports, and sends them to the server upon completion. Even at this stage, the emotion engine monitors the user's state and analyzes stress and satisfaction levels.

[1790] Step 6:

[1791] The server receives the final versions of proposals and reports, along with sentiment data, and stores them in a central database. The server analyzes the received data and sends notifications to relevant parties. This facilitates the management of proposals and reports and the provision of sentiment feedback.

[1792] Step 7:

[1793] The user enters market research questions into a terminal and sends them to the server. When creating the survey, the emotion engine analyzes the user's state and generates emotion data. The server receives and stores the questions and emotion data.

[1794] Step 8:

[1795] The server generates a survey link and sends it to each participant based on the distribution list. During this sending process, the sentiment engine analyzes user responses. The collected survey response data is temporarily stored on the server.

[1796] Step 9:

[1797] The user enters their video shooting plan into the terminal and sends the completed video to the server. The terminal edits the video and temporarily saves the final version. During shooting, an emotion engine monitors the user's state and analyzes emotional data.

[1798] Step 10:

[1799] The server receives the completed video and sentiment data and stores them in a central database. The server generates the necessary links and provides them to the user. This allows for the management of video content and sentiment feedback.

[1800] Step 11:

[1801] Store staff use smart glasses to scan customers' facial expressions. The device collects customer facial data in real time, and an emotion engine analyzes it to generate emotion data. Real-time emotion information of the customer when they visit the store is displayed on the device.

[1802] Step 12:

[1803] The server receives customer sentiment data and stores it in a central database along with past visit data and sentiment history. Based on the analysis results, the server sends notifications to store staff to provide the best possible service. This improves the customer experience.

[1804] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1805] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1806] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1807] [Fourth Embodiment]

[1808] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1809] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1810] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1811] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1812] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1813] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1814] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1815] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1816] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1817] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1818] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1819] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1820] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1821] This invention relates to a system for centrally managing and efficiently executing various tasks. This system enables the efficient operation of a wide range of tasks through the collaboration of users, terminals, and servers.

[1822] Daily schedule management

[1823] 1. Schedule entry:

[1824] Users enter their daily schedules and tasks from each device.

[1825] The terminal temporarily saves the entered schedule and sends it to the server.

[1826] 2. Schedule saving and synchronization:

[1827] The server stores the received schedule information in a central database.

[1828] The server synchronizes the latest schedule information to each terminal after saving it.

[1829] Meeting minutes

[1830] 1. Entering meeting minutes:

[1831] The user enters meeting minutes into their terminal during the meeting.

[1832] The terminal temporarily saves meeting minutes in real time.

[1833] 2. Sending and saving meeting minutes:

[1834] The user clicks the save button after the meeting ends.

[1835] The terminal sends the meeting minutes to the server.

[1836] The server saves the received meeting minutes to a central database.

[1837] Planning and report writing

[1838] 1. Planning and input:

[1839] Users create drafts of plans and reports using their devices.

[1840] The terminal provides users with necessary templates and historical documents.

[1841] 2. Saving and Notifications:

[1842] The terminal sends completed plans and reports to the server.

[1843] The server stores them in a central database and sends notifications to the relevant parties.

[1844] Market research

[1845] 1. Creating and preparing the questionnaire:

[1846] The user sets the market research questions on their device.

[1847] The terminal sends the configured question to the server.

[1848] 2. Distribution of questionnaires:

[1849] The server generates a survey link and sends it to each recipient based on the distribution list.

[1850] 3. Response collection and analysis:

[1851] Participants will answer the questionnaire via the link they receive.

[1852] The device sends the response data to the server.

[1853] The server analyzes the collected response data, generates a visual report, and provides it to the user.

[1854] Employee training

[1855] 1. Curriculum management and delivery:

[1856] The server manages the employee training curriculum and distributes training modules to each terminal.

[1857] 2. Report on educational progress:

[1858] The terminal reports the progress of the educational module to the server in real time.

[1859] Users enter feedback after each session is completed and send it to the server.

[1860] Creating a promotional video

[1861] 1. Planning and developing video content:

[1862] The user uses the device to plan the video content and shooting schedule.

[1863] 2. Filming and editing:

[1864] The terminal uses video editing software to edit the footage and sends the final version to the server.

[1865] The server stores the completed video and generates the necessary links to provide to the user.

[1866] System testing and monitoring

[1867] 1. Run the test:

[1868] The terminal executes the test script and sends the results to the server.

[1869] 2. Log collection and monitoring:

[1870] The server collects operation logs from each terminal and monitors for anomalies.

[1871] The server will report any problems detected to the user.

[1872] Call center initial support

[1873] 1. Inquiry handling:

[1874] The operator enters the customer's inquiry into the terminal.

[1875] 2. Recording and Analysis:

[1876] The server stores the query content in a central database and performs analysis.

[1877] Specific example:

[1878] For example, when a user uses the "market research" function, the process proceeds as follows:

[1879] 1. The user sets the survey questions on their device and submits them.

[1880] 2. The server receives the questions and generates a survey link.

[1881] 3. The device sends the link to the person being surveyed.

[1882] 4. Participants will answer the questionnaire using the provided link.

[1883] 5. The device sends the response data to the server.

[1884] 6. The server aggregates the data, generates a report including the analysis results, and provides it to the user.

[1885] This allows the system to help users conduct market research effectively.

[1886] The following describes the processing flow.

[1887] Market research processing steps

[1888] Creating and preparing the questionnaire for distribution

[1889] Step 1:

[1890] The user enters market research survey questions from their device.

[1891] The terminal temporarily stores the entered questions in local storage and checks the integrity of the input.

[1892] Step 2:

[1893] The terminal sends the questions that have completed the integrity check to the server.

[1894] The server reviews the received questions and saves them to the central database.

[1895] Distribution of questionnaires

[1896] Step 3:

[1897] The server generates a link to the survey and emails the link to each survey participant based on the distribution list.

[1898] Step 4:

[1899] The terminal reports the transmission status to the server in real time.

[1900] The server confirms that the survey link has been successfully sent and updates the log.

[1901] Collection of responses

[1902] Step 5:

[1903] Participants click the received link to access the survey page.

[1904] The device displays the survey response screen, and the survey participants answer the questions.

[1905] Step 6:

[1906] The device temporarily stores the survey participants' responses in local storage and sends the data to the server once all responses have been completed.

[1907] Step 7:

[1908] The server saves the received response data to a central database and returns a confirmation message to the terminal acknowledging the successful saving.

[1909] Data aggregation and analysis

[1910] Step 8:

[1911] The server aggregates the stored response data and performs statistical analysis.

[1912] The server converts the analysis results into graphs and charts, generating visual reports.

[1913] Step 9:

[1914] The server sends the generated report to the terminal.

[1915] Users can view reports from their devices and provide feedback as needed.

[1916] Record of feedback

[1917] Step 10:

[1918] The device sends the user's input to the server.

[1919] The server saves the received feedback to a database and records it as an area for improvement in future market research.

[1920] The above processing steps enable the system to efficiently carry out the entire market research process, from data collection and analysis to reporting, in a consistent manner.

[1921] (Example 1)

[1922] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1923] Currently, there is a need for a centralized management system to efficiently handle multiple functions in business management and operations, such as schedule management, meeting minute creation, planning and report creation, market research, employee training, promotional video production, system testing and monitoring, and call center inquiry handling. When these tasks are managed individually, data synchronization and sharing become cumbersome, leading to decreased efficiency. Furthermore, there is a growing need for a system that reduces user effort while enabling accurate and rapid data processing.

[1924] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1925] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for terminals to temporarily store schedules and send them to the server, and means for the server to store schedule information in a central database and synchronize it with each terminal. This enables centralized management of schedule information and real-time data synchronization. It also includes means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to create plans and reports, save them after completion and notify relevant parties, and means for users to set market research questions, distribute questionnaires, and collect and analyze response data. The server also includes means for generating questionnaire links and sending them to each target based on a distribution list, means for managing employee training curricula, distributing training modules and reporting progress, means for users to formulate and edit video content and shooting plans and save the final version to the server, means for the server to collect system operation logs, detect and report problems, and means for call center operators to input and record inquiry details. This enables efficient and accurate centralized management of multiple tasks, significantly reducing user effort and improving overall operational efficiency.

[1926] A "user" refers to an entity that uses the system to perform various tasks.

[1927] A "terminal" refers to a device operated by a user, such as a computer or smartphone used for tasks like entering schedules, creating meeting minutes, or designing surveys.

[1928] A "server" refers to a central computer system used to store data and manage communication and data synchronization between terminals.

[1929] A "central database" refers to a database built on a server that centrally manages and stores all data generated within the system.

[1930] "Schedule information" refers to data about appointments and tasks entered by the user.

[1931] "Meeting minutes" refers to a document that users record during a meeting, including details of the meeting, discussions, and decisions made.

[1932] "Plans" and "reports" refer to plans and reports created by users, which are documents containing information about the business or project.

[1933] "Market research" refers to the process of collecting market reactions and opinions through surveys set by users.

[1934] A "survey link" is a URL generated by the server and sent to the survey participant, used to access the survey form.

[1935] An "employee training curriculum" refers to an educational program aimed at improving employees' skills and acquiring knowledge.

[1936] An "education module" is a part of the employee training curriculum and refers to the specific content of each lesson or session.

[1937] "Video content" refers to the scenario and structure that the user develops when creating a promotional video.

[1938] A "shooting plan" refers to a specific plan for shooting a video, including the shooting schedule, locations, and necessary equipment.

[1939] "Operation log" refers to data that records the operating status of a system or terminal.

[1940] A "call center operator" refers to a person who handles customer inquiries at a call center.

[1941] This invention relates to a system for centrally managing and efficiently executing various tasks. Through collaboration between users, terminals, and servers, it enables the efficient operation of a wide range of tasks.

[1942] Hardware and software to be used

[1943] hardware

[1944] Devices: Laptops, smartphones (iOS / Android), tablets (iPad), desktop PCs

[1945] Server: Cloud servers (AWS, Azure, Google Cloud)

[1946] software

[1947] Database management systems (PostgreSQL, MySQL)

[1948] Text editor

[1949] Video editing software

[1950] Communication method (Firebase Cloud Messaging, HTTPS)

[1951] Email sending system (SendGrid)

[1952] Analysis tools (Python Pandas, Matplotlib)

[1953] The specific embodiments of each function of this system are described below.

[1954] Schedule management

[1955] Input and data transmission

[1956] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[1957] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[1958] Data storage and synchronization

[1959] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[1960] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[1961] Market research

[1962] Questionnaire creation and distribution

[1963] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[1964] The terminal POSTs the question data to the server in JSON format.

[1965] Generate and send survey link

[1966] The server uses a URL shortening service to generate the survey link.

[1967] The server will send this link to the recipients using the email sending system (SendGrid).

[1968] Response collection and analysis

[1969] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[1970] The device temporarily stores the response data and sends it to the server in JSON format.

[1971] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[1972] Example of a prompt

[1973] Schedule management:

[1974] "Please add a 10:00 meeting to this week's schedule and save it to the server."

[1975] Market research:

[1976] "Please create a questionnaire about the new product and send it to the target audience."

[1977] The various functions of this system allow users to efficiently centralize and manage multiple tasks, significantly reducing effort and improving overall operational efficiency.

[1978] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1979] Daily schedule management

[1980] Step 1:

[1981] Users enter their daily schedules and tasks using a calendar app on each device. For example, they might enter "Meeting with a client at 10:00."

[1982] Input: Date, time, meeting content (e.g., 10:00, meeting with a client)

[1983] Action: The user creates a new schedule in the calendar app and clicks the save button.

[1984] Output: Schedule data stored in the local cache

[1985] Step 2:

[1986] The terminal temporarily stores the entered schedule data in its local cache. It then POSTs the data to the server in JSON format.

[1987] Input: Schedule data (e.g., 10:00, Meeting with a client)

[1988] Operation: Save to local cache and convert to JSON format.

[1989] Output: POST request to the server

[1990] Step 3:

[1991] The server parses the received schedule data and saves it to a database. Specifically, it uses PostgreSQL.

[1992] Input: Schedule data in JSON format

[1993] Operation: Data analysis and insertion into a PostgreSQL database.

[1994] Output: Schedule data stored in the database

[1995] Step 4:

[1996] The server sends real-time push notifications to each device with the latest schedule information using Firebase Cloud Messaging and other methods.

[1997] Input: Schedule data stored in the database

[1998] Operation: Triggering push notifications using Firebase Cloud Messaging

[1999] Output: Latest schedule notification received on each device

[2000] Market research

[2001] Step 1:

[2002] The user enters survey questions through a web interface on their laptop. The question "What do you think of the product?" is added.

[2003] Input: Question item (e.g., What do you think of the product?)

[2004] Operation: Enter and save a new question on the web interface.

[2005] Output: Question data stored in the local cache

[2006] Step 2:

[2007] The terminal POSTs the question data to the server in JSON format.

[2008] Input: Question data (e.g., What do you think of the product?)

[2009] Operation: Save to local cache and convert to JSON format.

[2010] Output: POST request to the server

[2011] Step 3:

[2012] The server uses a URL shortening service to generate the survey link.

[2013] Input: Question data

[2014] Operation: Generates survey links using a URL shortening service.

[2015] Output: Generated survey link

[2016] Step 4:

[2017] The server will send this link to the recipients using the email sending system (SendGrid).

[2018] Input: Generated survey link

[2019] Operation: Send emails based on distribution lists using SendGrid.

[2020] Output: Email and survey link sent to participants

[2021] Step 5:

[2022] Participants will click the link to access the survey form in their web browser, enter their answers, and submit it.

[2023] Input: Response data entered by survey participants.

[2024] How it works: Fill out the survey form in your web browser and submit your response.

[2025] Output: Sending response data to the server

[2026] Step 6:

[2027] The device temporarily stores the response data and sends it to the server in JSON format.

[2028] Input: Response data

[2029] Operation: Save to local cache and convert to JSON format.

[2030] Output: POST request to the server

[2031] Step 7:

[2032] The server stores the response data in a database and runs an analysis algorithm. It uses Python's Pandas and Matplotlib to perform the analysis and generate a visual report.

[2033] Input: Response data in JSON format

[2034] Function: Saves to database, runs analytical algorithms, generates visual reports (Pandas, Matplotlib)

[2035] Output: Visual reports displayed on the user's dashboard.

[2036] Example of a prompt

[2037] Schedule Management: "Add a 10:00 meeting to this week's schedule and save it to the server."

[2038] Market research: "Create a questionnaire about the new product and send it to the target audience."

[2039] This program's processing steps allow users to efficiently manage schedules and conduct market research, enabling them to manage various tasks in an integrated manner.

[2040] (Application Example 1)

[2041] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2042] Traditional business management systems have struggled to efficiently integrate scheduling, real-time tracking of operations, meeting minute creation, employee training, and system testing and monitoring within logistics centers. In particular, the inability to track the movement of goods and the progress of picking lists within the logistics center in real time led to decreased operational efficiency and increased errors. A new system is needed to address these challenges.

[2043] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[2044] In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database, means for users to input meeting minutes during a meeting and send them to the server after the meeting ends, means for users to set market research questions, distribute questionnaires, and collect and analyze response data, and means for tracking the movement of goods within the logistics center and the progress of picking lists in real time. This enables efficient operation of the entire logistics center and real-time management of various operations.

[2045] A "user" is an entity that uses a system to perform various tasks.

[2046] A "terminal" is a device used by a user to input information or utilize system functions.

[2047] A "central database" is a storage device used to centrally store information for the entire system.

[2048] A "server" is a computer system that receives and processes data transmitted from various terminals.

[2049] "Schedule information" refers to the daily schedules and tasks entered by the user.

[2050] "Meeting minutes" are documents that users record during a meeting and save afterward.

[2051] A "project" is a new project or plan created by a user.

[2052] A "report" is a document that summarizes the progress and results of a project.

[2053] "Market research" is the activity of collecting and analyzing data about a specific market.

[2054] A "questionnaire" is a survey conducted as part of market research, consisting of a series of questions.

[2055] "Data collection" refers to the act of gathering information transmitted from each device.

[2056] "Data analysis" is the process of verifying collected information and compiling it into a visual report.

[2057] An "employee training curriculum" is a learning program aimed at improving employees' skills.

[2058] An "education module" refers to software and teaching materials used to implement employee training curricula.

[2059] "Feedback" refers to the evaluations and opinions that users enter after completing an educational module.

[2060] A "promotional video" is video content produced for the purpose of advertising or public relations.

[2061] "Real-time tracking" is a function that allows you to instantly grasp the status of ongoing operations and logistics.

[2062] A "picking list" is a list that displays the items needed for a specific order or task.

[2063] This invention is a system for efficiently managing and operating operations at a logistics center. The specific program and its processing details for realizing this system are described below.

[2064] System Configuration

[2065] This system consists primarily of users, production terminals, an information processing server, and a central database. The terminals provide an interface for users to input information and send it to the server. The server stores the received information in the central database and synchronizes it with other terminals as needed.

[2066] Schedule management

[2067] Users input their daily tasks and schedules within the logistics center using terminals. This schedule information is temporarily stored on the terminals and then sent to the server. The server stores the received schedule information in a central database and synchronizes the latest schedule information with each terminal.

[2068] Real-time tracking

[2069] Data is entered via terminals to track the movement of goods within the logistics center and the progress of picking lists in real time. The terminals transmit this information to a server in real time, and the server stores it in a central database, thereby improving the efficiency of goods management.

[2070] Meeting minutes

[2071] During the meeting, users enter meeting minutes into their terminals. After the meeting ends, users click the save button, and the minutes are sent from their terminals to the server. The server saves the received minutes to a central database and sends notifications to the relevant parties.

[2072] Employee training

[2073] The server manages the employee training curriculum and delivers training modules to each terminal. Training progress reports are sent from the terminal to the server in real time. Users enter feedback after completing each session and send it to the server.

[2074] System testing and monitoring

[2075] The terminal collects system operation logs and sends them to the server. The server stores the operation logs of each terminal in a central database and reports any abnormalities to the user.

[2076] Specific example

[2077] For example, to add a new inventory check task, the user would enter the following information on the schedule input screen:

[2078] Example of a schedule input prompt

[2079] Please add a new task.

[2080] User: employee1

[2081] Task name: Inventory check

[2082] Date: 2023-10-10

[2083] Time: 14:00

[2084] In this way, the system enables efficient operation of the entire logistics center and real-time management of various tasks. Furthermore, it can streamline other incidental tasks such as schedule management and meeting minute creation.

[2085] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2086] Step 1:

[2087] The user opens the schedule entry screen using their device and enters their daily tasks and appointments. The entered data includes the username, task name, date, and time. The device temporarily saves the entered data and then sends it to the server.

[2088] Step 2:

[2089] The server receives schedule information sent from the terminals. The data includes the username, task name, date, and time, and this data is stored in a central database. After the schedule information is stored, the server sends a request to each terminal to synchronize with the latest schedule information.

[2090] Step 3:

[2091] Users use terminals to input information about the movement of items within the logistics center and picking lists in real time. The entered data is immediately sent from the terminal to the server.

[2092] Step 4:

[2093] The server receives real-time tracking data sent from the terminal and stores it in a central database. Based on the received data, the server monitors the movement of items and the progress of picking lists in real time.

[2094] Step 5:

[2095] During a meeting, the user enters meeting minutes using their device. After the meeting ends, the user clicks the save button on their device. The device temporarily saves the meeting minutes and sends them to the server.

[2096] Step 6:

[2097] The server receives meeting minutes data sent from terminals and stores it in a central database. This data includes the meeting title, agenda, participants, and minutes content. The server notifies relevant parties that the minutes have been saved.

[2098] Step 7:

[2099] Users access the employee training curriculum from their devices and utilize the training modules. After completing each session, users provide feedback. The device then sends the feedback and progress report to the server.

[2100] Step 8:

[2101] The server receives progress data and feedback on educational modules sent from terminals and stores them in a central database. The server manages educational progress and feedback in real time.

[2102] Step 9:

[2103] The terminal collects system operation status as logs and periodically sends them to the server. The log data includes operational anomalies, error messages, and performance metrics.

[2104] Step 10:

[2105] The server receives operation logs sent from the terminal and stores them in a central database. The server analyzes the log data and issues a warning to the user if an anomaly is detected.

[2106] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2107] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[2108] Emotional engine integration

[2109] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[2110] Daily schedule management

[2111] 1. Schedule entry:

[2112] Users enter their daily schedules and tasks from each device.

[2113] The terminal temporarily stores the entered schedule information and sends it to the server.

[2114] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data, which is then sent to the server.

[2115] 2. Schedule saving and synchronization:

[2116] The server stores schedule information and sentiment data in a central database.

[2117] The server synchronizes the latest schedule information and emotional feedback to each terminal.

[2118] Meeting minutes

[2119] 1. Entering meeting minutes:

[2120] The user enters meeting minutes into their terminal during the meeting.

[2121] The terminal temporarily saves meeting minutes in real time.

[2122] The emotion engine recognizes the user's emotions while they are typing and detects tension and stress.

[2123] 2. Sending and saving meeting minutes:

[2124] The user clicks the save button after the meeting ends.

[2125] The terminal sends meeting minutes and sentiment data to the server.

[2126] The server stores the received meeting minutes and sentiment data in a central database.

[2127] Planning and report writing

[2128] 1. Planning and input:

[2129] Users create drafts of plans and reports using their devices.

[2130] The terminal provides users with necessary templates and historical documents.

[2131] The emotion engine analyzes whether the user is experiencing stress or satisfaction.

[2132] 2. Saving and Notifications:

[2133] The terminal sends completed plans and reports to the server.

[2134] The server stores them in a central database and sends notifications to the relevant parties.

[2135] The emotion engine provides feedback that responds to the user's emotions.

[2136] Market research

[2137] 1. Creating and preparing the questionnaire:

[2138] The user sets the market research questions on their device.

[2139] The terminal sends the configured question to the server.

[2140] 2. Distribution of questionnaires:

[2141] The server generates the survey link and sends the link to each survey participant based on the distribution list.

[2142] The emotion engine analyzes the user's reaction when a link is sent.

[2143] 3. Response collection and analysis:

[2144] Participants click the received link to access the survey page.

[2145] The device displays the survey response screen, and the survey participants answer the questions.

[2146] The device sends the response data to the server.

[2147] The server analyzes the collected response data and generates a visual report.

[2148] The emotion engine analyzes emotional tendencies based on aggregated data and reflects them in the results.

[2149] Employee training

[2150] 1. Curriculum management and delivery:

[2151] The server manages the employee training curriculum and distributes training modules to each terminal.

[2152] The emotion engine adjusts the curriculum based on employee emotional data.

[2153] 2. Report on educational progress:

[2154] The terminal reports the progress of the educational module to the server in real time.

[2155] Users enter feedback after each session is completed and send it to the server.

[2156] The server receives feedback and sentiment data and analyzes the educational effect.

[2157] Creating a promotional video

[2158] 1. Planning and developing video content:

[2159] The user uses the device to plan the video content and shooting schedule.

[2160] The emotion engine analyzes the user's emotions and reflects them in the video content.

[2161] 2. Filming and editing:

[2162] The terminal uses video editing software to edit the footage and sends the final version to the server.

[2163] The server stores the completed video and generates the necessary links to provide to the user.

[2164] System testing and monitoring

[2165] 1. Run the test:

[2166] The terminal executes the test script and sends the results to the server.

[2167] 2. Log collection and monitoring:

[2168] The server collects operation logs from each terminal and monitors for anomalies.

[2169] The emotion engine evaluates the user's emotions while using the system and optimizes the user experience.

[2170] Call center initial support

[2171] 1. Inquiry handling:

[2172] The operator enters the customer's inquiry into the terminal.

[2173] The emotion engine analyzes the operator's emotions during interactions and provides advice for stress reduction.

[2174] 2. Recording and Analysis:

[2175] The server stores the query content in a central database and performs analysis.

[2176] The emotion engine combines collected data with emotional data to evaluate the quality of customer service.

[2177] Specific example:

[2178] For example, when a user uses the "market research" function, the process proceeds as follows:

[2179] 1. The user inputs market research questions from their device. The emotion engine analyzes the user's facial expressions and voice as they input the information.

[2180] 2. The terminal sends the entered questions to the server. The server stores the questions and sentiment data in a central database.

[2181] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[2182] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[2183] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[2184] This allows the system to help users conduct market research effectively and to provide higher quality services by utilizing sentiment data.

[2185] The following describes the processing flow.

[2186] Processing steps in market research

[2187] Creating and preparing the questionnaire for distribution

[2188] Step 1:

[2189] The user enters market research questions from their terminal.

[2190] The device temporarily stores the entered questions and sends them to the emotion engine.

[2191] The emotion engine analyzes the user's facial expressions and voice during input to generate emotion data.

[2192] The emotion engine sends emotional data back to the device.

[2193] Step 2:

[2194] The device sends the questionnaire items and sentiment data to the server.

[2195] The server stores the questionnaire items and sentiment data in a central database.

[2196] Distribution of questionnaires

[2197] Step 3:

[2198] The server generates a survey link and sends the link via email to each survey participant based on the distribution list.

[2199] Step 4:

[2200] The terminal reports the transmission status to the server in real time.

[2201] The server logs the successful submission of the survey link.

[2202] The emotion engine re-analyzes the user's response during this process and updates the emotion data.

[2203] Collection of responses

[2204] Step 5:

[2205] Participants in the survey will access the questionnaire page by clicking the link they receive.

[2206] The device displays the survey response screen, and the survey participants answer the questions.

[2207] Step 6:

[2208] The device temporarily stores the survey participants' responses and sends the data to the server once all responses have been completed.

[2209] Step 7:

[2210] The server stores the response data in a central database.

[2211] The emotion engine analyzes the emotional data of the survey participants on the server and stores it together with the data in the database.

[2212] The server sends a confirmation message to the terminal acknowledging that the save was successful.

[2213] Data aggregation and analysis

[2214] Step 8:

[2215] The server aggregates the stored response data and performs statistical analysis.

[2216] The emotion engine works with the server to analyze emotional tendencies based on aggregated data and reflect them in the results.

[2217] Step 9:

[2218] The server converts the analysis results into graphs and charts, generating visual reports.

[2219] The server sends the generated report and sentiment analysis results to the terminal.

[2220] Step 10:

[2221] Users can view reports from their devices and provide feedback as needed.

[2222] The device sends user feedback to the emotion engine.

[2223] The emotion engine analyzes feedback and emotional data to derive areas for improvement for the next market research.

[2224] The emotion engine sends the analysis results to the server and stores them in the database.

[2225] This allows the system to efficiently manage the entire market research process and leverage user sentiment data to provide high-quality feedback and analytical results.

[2226] (Example 2)

[2227] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2228] Traditional business management systems focus on individual tasks and schedule management, but lack mechanisms to provide feedback that takes into account the user's emotional state. Furthermore, few systems utilize emotional data to improve work efficiency and quality. As a result, there is no effective method for managing user stress and satisfaction, making it a challenge to improve work quality and employee mental health.

[2229] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and store it in a central database along with emotional data; means for users to input meeting minutes during a meeting, perform emotional analysis, and send them to the server after the meeting ends; and means for users to create plans and reports, receive emotional feedback, and then save them and notify relevant parties after completion. This enables the analysis of users' emotional data in real time and provides optimal feedback based on that analysis, thereby improving work efficiency and the mental health of employees.

[2230] "Schedule information" refers to appointments and action plans that users enter to manage their daily work and tasks.

[2231] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions, voice, behavioral patterns, and other factors.

[2232] A "central database" is a database used to centrally manage and store data for the entire system.

[2233] Meeting minutes are documents that record the content discussed and decisions made during a meeting.

[2234] "Emotion analysis" is the process of recognizing a user's emotional state based on their input and actions.

[2235] "Planning" refers to a concrete plan or concept for achieving a specific objective.

[2236] A "report" is a document that summarizes the progress and results of work or a project.

[2237] A "survey" is a method of collecting responses to questions set up for a specific research purpose.

[2238] "Employee training" refers to the provision of training and curricula designed to improve the skills and knowledge of employees and staff.

[2239] An "educational module" is a unit of learning content designed for a specific educational purpose.

[2240] "Video content" refers to the content of videos filmed for purposes such as promotion, explanation, or education.

[2241] A "shooting plan" is a plan that is formulated in advance to ensure efficient video shooting.

[2242] An "operation log" refers to a record of operations and events performed by a system or terminal.

[2243] A "call center" is a department or organization that functions as a point of contact for handling customer inquiries.

[2244] "Inquiry details" refers to the specific content of questions and consultations received by customers at the call center.

[2245] "Stress management" refers to methods for evaluating the stress that users and employees experience at work and appropriately mitigating it.

[2246] A "visual report" is a report that presents data in a visually easy-to-understand format, such as graphs and charts.

[2247] This invention relates to a system for centrally managing and efficiently executing various tasks, and further incorporates an emotion engine that recognizes user emotions in real time and provides optimal feedback based on those emotions. This system is realized through the collaboration of the user, terminal, server, and emotion engine.

[2248] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. The emotional information recognized by the emotion engine is sent to the server and used in various business processes.

[2249] Hardware and software to be used

[2250] Emotion engine: Uses Microsoft Azure Cognitive Services or Google Cloud Emotion AI.

[2251] Servers: We will use AWS EC2 servers and Amazon RDS as the database service.

[2252] Devices: iOS / Android devices are accepted.

[2253] Video editing software: Adobe Premiere Pro will be used.

[2254] Log collection software: Use the ELK stack (Elasticsearch, Kibana, Logstash).

[2255] Emotional engine integration

[2256] The emotion engine analyzes user input, behavioral patterns, voice, and facial expressions to recognize the user's emotions. This recognized emotion information is sent to the server and used in various business processes.

[2257] Daily schedule management

[2258] Users input their daily schedules and tasks from their respective devices, and their facial expressions and voices are analyzed by an emotion engine. The devices temporarily store the entered schedule information and send it to the server along with the emotion data. The server stores the schedule information and emotion data in a central database and synchronizes the latest schedule information and emotion feedback to each device.

[2259] Meeting minutes

[2260] During a meeting, the user enters meeting minutes into a terminal. The emotion engine recognizes the user's emotions while they are typing, detecting tension and stress. When the user clicks the save button after the meeting ends, the terminal sends the meeting minutes and emotion data to the server, which stores them in a central database.

[2261] Planning and report writing

[2262] Users create drafts of projects and reports using a terminal. The terminal provides users with necessary templates and past materials, and the emotion engine analyzes whether the user is experiencing stress or satisfaction. Completed projects and reports are sent to a server, stored in a central database, and notifications are sent to relevant parties. The emotion engine provides feedback tailored to the user's emotions.

[2263] Market research

[2264] The user sets market research questions on their device and sends the set questions to the server. The server generates a survey link and sends the link to each research participant based on the distribution list. The emotion engine analyzes the user's response during distribution. Research participants answer the survey via the link, and the data is sent from their device to the server. The server analyzes the collected response data and generates a visual report. The emotion engine uses this aggregated data to analyze emotional trends and reflects them in the results.

[2265] Creating a promotional video

[2266] The user uses a device to plan the video content and shooting schedule, and an emotion engine analyzes the user's emotions and incorporates them into the video content. The device then uses video editing software to edit the footage and sends the final version to the server. The server stores the completed video, generates the necessary links, and provides them to the user.

[2267] System testing and monitoring

[2268] The terminal executes a test script and sends the results to the server. The server collects operation logs from each terminal and monitors for anomalies. The sentiment engine evaluates the user's emotions while using the system and optimizes the user experience.

[2269] Call center initial support

[2270] Operators input customer inquiries into a terminal, and an emotion engine analyzes the operator's emotions during the interaction, providing stress reduction advice. The server stores the inquiry details in a central database and combines them with emotion data to evaluate the quality of customer service.

[2271] Examples of specific cases and prompt statements

[2272] For example, when a user uses the "market research" function, the process proceeds as follows:

[2273] 1. Users input market research questions from their devices, and an emotion engine analyzes the user's facial expressions and voice as they input the information.

[2274] 2. The terminal sends the entered questions to the server, and the server stores the questions and sentiment data in a central database.

[2275] 3. The server generates a survey link and sends it to each survey participant based on the distribution list. The sentiment engine analyzes user responses at the time of sending.

[2276] 4. Participants answer the questionnaire via a link, and their devices send the data to the server. The server stores the response data, and an emotion engine analyzes the emotional tendencies and reflects them in the results.

[2277] 5. The server generates a visual report and sends it to the terminal. The user reviews the report, and the terminal sends feedback to the server. The sentiment engine uses the feedback and sentiment data to reflect areas for improvement in the next survey.

[2278] Example of a prompt:

[2279] "Based on the analysis of sentiment data from the following market research questions, please tell us what improvements can be made."

[2280] In this way, the system helps users conduct market research effectively and provides higher quality services by utilizing sentiment data.

[2281] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2282] Step 1: Enter schedule

[2283] 1.1 Users input their daily schedules and tasks from their respective devices. Input includes items such as "meetings," "document creation," and "lunch meetings."

[2284] 1.2 The terminal temporarily saves the entered schedule information. The input is the schedule information entered by the user.

[2285] 1.3 The emotion engine analyzes the user's facial expressions and voice during input and generates emotion data. Input data comes from the user's camera and microphone.

[2286] 1.4 The terminal sends schedule information and sentiment data to the server. The output includes schedule information and sentiment data.

[2287] 1.5 The server saves schedule information and sentiment data to the central database. Schedule information and sentiment data are saved to the central database as output.

[2288] Step 2: Schedule synchronization

[2289] 2.1 The server synchronizes the latest schedule information and sentiment feedback to each terminal. The input is information stored in a central database, and the output is updated data sent to the terminal.

[2290] 2.2 The device locally stores the schedule information and sentiment feedback it receives. A specific example is push notifications for a scheduling app.

[2291] Step 3: Enter meeting minutes

[2292] 3.1 Users enter meeting minutes into their terminals during the meeting. This input includes information such as "agenda," "decisions," and "next actions."

[2293] 3.2 The terminal temporarily saves meeting minutes in real time. The input is the meeting minutes information entered by the user, and the output is the temporarily saved data.

[2294] 3.3 The emotion engine recognizes the user's emotions during input and detects tension and stress. The user's facial expression data is the input, and emotion data is generated as the output.

[2295] Step 4: Send and save meeting minutes

[2296] 4.1 The user clicks the save button after the meeting ends. User actions are included as input.

[2297] 4.2 The terminal sends meeting minutes and sentiment data to the server. Meeting minutes and sentiment data are sent to the server as output.

[2298] 4.3 The server saves the received meeting minutes and sentiment data to the central database. The output includes the meeting minutes and sentiment data stored in the central database.

[2299] Step 5: Planning and Report Creation

[2300] 5.1 Users create drafts of plans and reports using their devices. The input includes the content of the plan.

[2301] 5.2 The terminal provides users with necessary templates and historical documents. Related documents are displayed to the user as output.

[2302] 5.3 The emotion engine analyzes whether the user is experiencing stress or satisfaction. Inputs include the user's keystroke patterns and mouse movements, and emotional data is generated as output.

[2303] 5.4 The terminal sends the completed project proposal or report to the server. The project proposal or report is sent to the server as output.

[2304] 5.5 The server stores them in a central database and sends notifications to the relevant parties. The output includes the stored data and the sent notifications.

[2305] 5.6 The emotion engine provides feedback that corresponds to the user's emotions. The feedback is displayed to the user as output.

[2306] Step 6: Market Research

[2307] 6.1 The user sets the market research questions on the device. The questions are included as input.

[2308] 6.2 The terminal sends the configured questions to the server. The question items are sent to the server as output.

[2309] 6.3 The server generates the survey link and sends the link to each survey participant based on the distribution list. The generated link and the link itself are sent as output.

[2310] 6.4 The emotion engine analyzes the user's reactions. User facial expression data is taken as input, and analyzed emotion data is generated as output.

[2311] 6.5 Participants answer the questionnaire via a link, and their devices send the data to the server. The response data is included as input.

[2312] 6.6 The server stores the collected response data, and the sentiment engine analyzes the sentiment trends. The output includes the stored response data and the analysis results.

[2313] Step 7: Create a promotional video

[2314] 7.1 The user uses a device to formulate the video content and shooting plan. The video content is included as input.

[2315] 7.2 The emotion engine analyzes the user's emotions and reflects them in the video content. User emotion data is the input, and it is reflected in the video content as the output.

[2316] 7.3 The terminal uses video editing software to edit the footage and sends the final version to the server. The edited video is sent to the server as output.

[2317] 7.4 The server stores the completed video and generates the necessary links to provide to the user. The links are generated and provided as output.

[2318] Step 8: System Testing and Monitoring

[2319] 8.1 The terminal executes the test script and sends the results to the server. The test results are included as input.

[2320] 8.2 The server collects operation logs from each terminal and monitors for anomalies. Operation logs are included as input.

[2321] 8.3 The emotion engine evaluates the user's emotions while using the system and optimizes the user experience. The input is user emotion data, and the output is optimized feedback.

[2322] Step 9: Temporary support at the call center

[2323] 9.1 An operator enters a customer inquiry into a terminal, and the emotion engine analyzes the response. The input includes the content of the inquiry.

[2324] 9.2 The terminal sends the entered query content to the server. The query data is sent to the server as output.

[2325] 9.3 The server stores the inquiry details in a central database and evaluates the quality of customer service by combining them with sentiment data. The output includes the stored data and the evaluation results.

[2326] (Application Example 2)

[2327] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2328] In modern brick-and-mortar stores, much effort is being made to improve customer satisfaction, and as part of this, it is necessary to accurately understand customer emotions and provide services based on those emotions. However, with previous systems, it was difficult to recognize customer emotions in real time and provide optimal feedback based on that information. Furthermore, it was difficult to centrally manage customer emotion data and use it to help with future visits. As a result, customer satisfaction tended to decline and the quality of service varied.

[2329] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for users to input schedule information from each terminal and save it in a central database; means for users to input meeting minutes during a meeting and send them to the server after the meeting ends; means for users to create plans and reports, save them after completion, and notify relevant parties; means for users to set market research questions, distribute questionnaires, and collect and analyze response data; means for managing employee training curricula, distributing training modules, and reporting progress; means for users to formulate video content and shooting plans, edit them, and save the final version to the server; means for collecting system operation logs, detecting and reporting problems; means for inputting and recording inquiry content to call center operators; means for recognizing customer emotional information in real time and providing store staff with the most appropriate service according to that emotional state; and means for saving past visit data and emotional history and providing the most appropriate service on the next visit. This makes it possible to recognize customer emotional states in real time and provide the most appropriate service based on that information, thereby improving customer satisfaction and standardizing service quality.

[2330] A "user" refers to a person who uses a system to input, manipulate, and manage various types of information.

[2331] A "terminal" refers to a device used by a user to input and manipulate information. Examples include computers, smartphones, and tablets.

[2332] A "server" refers to a central system that receives, stores, and processes information sent by users.

[2333] A "central database" refers to a large-scale database where information managed by a system is stored centrally.

[2334] An "emotion engine" refers to a system that analyzes a user's facial expressions, voice, and behavioral patterns to recognize their emotions in real time.

[2335] "Schedule information" refers to information about appointments and tasks entered by the user.

[2336] "Meeting" refers to a conference or meeting where users enter meeting minutes.

[2337] "Meeting minutes" refers to a document that records the content discussed and decisions made during a meeting.

[2338] "Planning" refers to a plan or concept for achieving a specific objective.

[2339] A "report" refers to a document that summarizes the results of research and analysis on a specific theme or topic.

[2340] "Market research" refers to research activities conducted to understand consumer needs and market trends.

[2341] A "survey" refers to a method used in market research to distribute questions to target individuals and collect their responses.

[2342] "Employee training" refers to educational programs implemented with the aim of improving employees' skills and knowledge.

[2343] An "educational module" refers to an individual learning unit used for employee training.

[2344] "Video content" refers to information or plans that are recorded as video.

[2345] A "filming plan" refers to the plan and schedule for shooting the content of a video.

[2346] "Operation log" refers to the operation history and event records of a system or terminal.

[2347] A "call center" refers to a department that has a customer service function, handling inquiries and providing support.

[2348] An "operator" refers to a person who handles customer inquiries in a call center.

[2349] A "customer" refers to a person who visits a physical store and receives services.

[2350] "Visit data" refers to information about when a customer visits a store.

[2351] "Service" refers to the benefits and assistance that a store provides to its customers.

[2352] The following describes the specific mechanism and processing of the system as an embodiment of this invention. This system is realized through the collaboration of a user, a terminal, a server, and an emotion engine.

[2353] Program generation and processing flow

[2354] Hardware and software usage

[2355] Hardware: User devices include smartphones, tablets, smart glasses, and head-mounted displays. Terminals used by store staff, as well as cameras and sensors for analyzing emotional data, will also be used.

[2356] Software: Python, OpenCV, and an emotion recognition library (e.g., EmotionRecognition) are used. This allows for real-time recognition of customer facial expressions and voices by analyzing camera footage, and generates emotion data.

[2357] User

[2358] Users input various data such as schedule information, meeting minutes, plans, reports, and survey settings through their devices and send it to the server. In physical stores, emotional information of users when they visit as customers is also collected. For example, store staff using smart glasses scan customers' facial expressions and analyze that data.

[2359] terminal

[2360] The terminal temporarily stores user-entered data and sends it to the server. Furthermore, it analyzes emotional data collected through the emotion engine, monitors the results in real time, and provides appropriate feedback. In physical stores, it analyzes camera footage to understand customer emotions. It also manages past visit data and emotional history to provide optimal service for subsequent visits.

[2361] server

[2362] The server stores schedule information, meeting minutes, plans, reports, survey responses, training module progress, activity logs, and sentiment data submitted by users in a central database. It also synchronizes the latest information with each terminal and provides necessary notifications. In physical stores, it analyzes customer sentiment data in real time and supports the provision of optimal services based on the results.

[2363] Specific example

[2364] Consider a scenario where store staff at a physical store use smart glasses to analyze customers' emotions in real time. If the staff member detects that the customer is stressed, a notification will appear on their device saying, "Please provide a relaxing service." Furthermore, if there is data indicating that the customer has previously been relaxed, the system will suggest the most suitable service for that customer.

[2365] Example of a prompt

[2366] "We are developing an application that uses new technology to recognize customer emotions in real time and provide optimal feedback to store staff. Specifically, it's a system that analyzes camera footage to predict customer emotions and notifies staff of the results. Could you please provide a detailed code example for this system?"

[2367] Thus, this invention aims to improve customer service in physical stores by working in conjunction with an emotion engine.

[2368] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2369] Step 1:

[2370] The user enters schedule information into the terminal. The terminal temporarily stores the entered schedule information and sends it to the server. This schedule information includes the date, time, and details of the appointment. The entered data is temporarily stored on the terminal and sent to the server.

[2371] Step 2:

[2372] The server receives schedule information sent by users and stores it in a central database. The server analyzes the received schedule information and synchronizes the latest schedule information to each terminal. This ensures that all devices have the most up-to-date schedule information.

[2373] Step 3:

[2374] During a meeting, the user enters meeting minutes into a terminal. The terminal temporarily stores the entered minutes in real time and sends them to the server after the meeting ends. At this stage, the emotion engine analyzes the user's facial expressions and voice to generate emotion data. The meeting minutes and emotion data are temporarily stored on the terminal.

[2375] Step 4:

[2376] The server receives meeting minutes and sentiment data after meetings and stores them in a central database. The server then analyzes the stored data and generates necessary feedback. This process manages both the meeting minutes and sentiment data.

[2377] Step 5:

[2378] Users input plans and reports into a terminal, and upon completion, save them and notify relevant parties. The terminal temporarily stores drafts of the plans and reports, and sends them to the server upon completion. Even at this stage, the emotion engine monitors the user's state and analyzes stress and satisfaction levels.

[2379] Step 6:

[2380] The server receives the final versions of proposals and reports, along with sentiment data, and stores them in a central database. The server analyzes the received data and sends notifications to relevant parties. This facilitates the management of proposals and reports and the provision of sentiment feedback.

[2381] Step 7:

[2382] The user enters market research questions into a terminal and sends them to the server. When creating the survey, the emotion engine analyzes the user's state and generates emotion data. The server receives and stores the questions and emotion data.

[2383] Step 8:

[2384] The server generates a survey link and sends it to each participant based on the distribution list. During this sending process, the sentiment engine analyzes user responses. The collected survey response data is temporarily stored on the server.

[2385] Step 9:

[2386] The user enters their video shooting plan into the terminal and sends the completed video to the server. The terminal edits the video and temporarily saves the final version. During shooting, an emotion engine monitors the user's state and analyzes emotional data.

[2387] Step 10:

[2388] The server receives the completed video and sentiment data and stores them in a central database. The server generates the necessary links and provides them to the user. This allows for the management of video content and sentiment feedback.

[2389] Step 11:

[2390] Store staff use smart g...

Claims

1. A means for users to input schedule information from each device and save it to a central database, A means for users to enter meeting minutes during a meeting and send them to the server after the meeting ends, A means for users to create plans and reports, save them after completion, and notify relevant parties, A means for users to set market research questions, distribute questionnaires, and collect and analyze response data, A means of managing employee training curricula, distributing training modules, and reporting on progress, A means for users to formulate video content and shooting plans, edit them, and save the final version to a server, A means of collecting system operation logs, detecting and reporting problems, A means for call center operators to input and record the content of inquiries, A system that includes this.

2. The system according to claim 1, wherein the server receives schedule information and automatically sends the latest schedule to each terminal.

3. The system according to claim 1, wherein the server aggregates the collected survey response data and generates a visual report.

Images