System

Abstract

A system is provided.SOLUTION: The system includes a means for creating a database dedicated to each company, a means for managing authentication information of each company, a means for storing and managing confidential data for each company, a means for accessing a common dedicated knowledge base, a means for uploading and managing voice and moving images, and a means for generating and managing billing information for each company.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, many small and medium-sized enterprises (SMEs) struggle with information management and sharing, and the risk of leaking confidential information is particularly high. Furthermore, a lack of personnel with specialized knowledge often hinders progress in improving business efficiency and digitalization. There is a demand for an environment that allows employees to utilize specialized knowledge while preventing data entered by employees from leaking to the outside. Furthermore, there is a lack of means to efficiently manage and utilize business audio and video data. The purpose of this invention is to solve these issues and support the digitalization and productivity improvement of SMEs. [Means for solving the problem]

[0005] The present invention provides a means for creating a database dedicated to each company, a means for managing authentication information for each company, and a means for storing and managing confidential data for each company. It also includes a means for accessing a common dedicated knowledge base and a means for uploading and managing audio and video. This allows for effective use of common expertise while preventing leaks of data for each company. It also has a means for generating and managing billing information for each company, supporting accurate billing processing based on usage history. It also provides a means for accepting user queries against the common dedicated knowledge base and providing relevant information. It also has a means for converting uploaded audio data into text and generating metadata for classification and search, thereby achieving business efficiency.

[0006] A "private company database" is a database designed to store and manage data for a specific company, and is kept confidential by being separated from data for other companies.

[0007] "Authentication information" refers to information used to verify the identity of a user accessing a system, including user IDs and passwords, and is used to prevent unauthorized access to the system.

[0008] "Confidential data" refers to information that is important to a company and that would be risky to disclose, such as financial data, customer information, and project details.

[0009] A "dedicated knowledge base" is a database that accumulates specialized knowledge about a specific field or business, and is a source of information that can be shared by multiple companies.

[0010] "Uploading audio and video" refers to the process in which a user records work content and know-how in audio or video format and saves the data in the system's storage.

[0011] "Metadata" is additional information that describes the content of audio and video data, and includes attribute information to facilitate searching and classification.

[0012] "Billing information" refers to billing details issued based on the content and volume of services used by the user (company), and includes monthly usage fees, customization fees, etc.

[0013] A "query" is an inquiry or question from a user to a database, and includes search conditions for obtaining required information.

[0014] "Converting uploaded audio data to text" refers to the process of automatically converting recorded audio data into written information, enabling text-based search and analysis.

[0015] "Classification" is the process of separating data into categories based on specific criteria, for the purpose of making data management and retrieval more efficient.

[0016] A "search function" is a function provided to quickly find specific information and capable of returning the results of a query against data in a database. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a generative AI platform for small and medium-sized enterprises, which supports the digitalization and productivity improvement of small and medium-sized enterprises by providing dedicated environments for each company and a common environment.

[0039] Overall system overview

[0040] The system provides each small and medium-sized enterprise with a dedicated database, safely manages confidential data for each company, and creates an environment where common expertise can be utilized. It also includes a variety of functions, such as managing audio and video data, sharing business know-how, and processing billing information.

[0041] Setting up a dedicated environment

[0042] 1. Server:

[0043] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[0044] Initialize the administrator account and provide credentials to the company.

[0045] 2. User (Administrator):

[0046] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[0047] 3. Server:

[0048] Manage employee account credentials and implement security measures to prevent unauthorized access.

[0049] Data entry and management

[0050] 1. User:

[0051] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[0052] Use a specific data entry form and click the "Save" button.

[0053] 2. Terminal:

[0054] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[0055] 3. Server:

[0056] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[0057] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[0058] Leveraging expertise in a common environment

[0059] 1. User:

[0060] Enter your tax and legal questions in a common environment and click the "Submit" button.

[0061] 2. Terminal:

[0062] The user's question is sent to a common environment and the relevant information is searched for.

[0063] 3. Server:

[0064] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0065] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[0066] Audio and video data management

[0067] 1. User:

[0068] Record and upload business content and know-how in audio or video format.

[0069] 2. Terminal:

[0070] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[0071] 3. Server:

[0072] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0073] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[0074] Manage your billing information

[0075] 1. Server:

[0076] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[0077] 2. Terminal:

[0078] Display billing information to administrators and provide payment links.

[0079] 3. Admin user:

[0080] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[0081] Example: At the end of each month, the server aggregates the usage history and generates the appropriate invoice, which the administrator can then view and easily complete the payment.

[0082] In this way, the present invention combines data management for each small and medium-sized enterprise with the utilization of common expertise to achieve efficient business operations and accurate billing processing using voice and video, thereby resolving many of the challenges faced by small and medium-sized enterprises and improving productivity.

[0083] The processing flow will be explained below.

[0084] Setting up a dedicated environment

[0085] Step 1:

[0086] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[0087] Step 2:

[0088] The server initializes the administrator account and provides the credentials to the company.

[0089] Step 3:

[0090] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[0091] Step 4:

[0092] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[0093] Data entry and management

[0094] Step 1:

[0095] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[0096] Step 2:

[0097] A user uses a particular data entry form and clicks the "Save" button.

[0098] Step 3:

[0099] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[0100] Step 4:

[0101] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[0102] Leveraging expertise in a common environment

[0103] Step 1:

[0104] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[0105] Step 2:

[0106] The terminal transmits the user's question to the common environment and searches for relevant information.

[0107] Step 3:

[0108] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0109] Audio and video data management

[0110] Step 1:

[0111] Users record and upload their business operations and know-how in audio and video formats.

[0112] Step 2:

[0113] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[0114] Step 3:

[0115] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0116] Manage your billing information

[0117] Step 1:

[0118] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[0119] Step 2:

[0120] The server generates an invoice and notifies the administrative user.

[0121] Step 3:

[0122] The terminal displays billing information to the administrator and provides a payment link.

[0123] Step 4:

[0124] Admin user reviews invoices and makes payments online.

[0125] Step 5:

[0126] The server updates the records in the system when the payment is completed.

[0127] Example 1

[0128] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] Currently, many small and medium-sized enterprises have not made much progress in digitalization, and are facing challenges in business efficiency and productivity. Furthermore, it is difficult to utilize the different data management and expertise that each company has, and in particular, the management of audio and video data and proper billing processing are insufficient. As a result, they are unable to obtain real-time information or appropriate advice for daily operations and management decisions, which is hindering productivity improvement.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0131] In this invention, the server includes a means for generating a database for storing data specific to each company, a means for managing authentication information for each company, a means for inputting, saving, and managing confidential data for each company, a means for accessing a common knowledge base and providing related information, a means for uploading, managing, and analyzing audio and video data, and a means for generating and managing billing information based on each company's usage history. This allows small and medium-sized enterprises to promote digitalization, obtain information in real time, and receive appropriate advice. It also improves business efficiency and accurate billing processing, significantly increasing productivity.

[0132] A "database" is a system for managing, storing, and efficiently manipulating a collection of data created specifically for each company.

[0133] "Authentication information" refers to information required for a user to access a system, and is primarily composed of a username, password, security token, and the like.

[0134] "Confidential data" refers to data that should only be used within a specific company, such as a company's sales data or customer information, and must be managed with high security.

[0135] A "knowledge base" is a database in which various specialized knowledge and information are organized and stored, and is used to provide appropriate answers to user questions.

[0136] "Audio data" is a recording of a user's work content or know-how, and is mainly saved in the form of an audio file.

[0137] "Video data" refers to work content or know-how recorded by a user, and is mainly saved in the form of a video file.

[0138] "Uploading" is the process by which a user sends data from their device to a server and stores it.

[0139] "Analysis" is the process of analyzing uploaded audio or video, converting the content into text, and generating metadata.

[0140] "Metadata" is auxiliary information added to facilitate data management and search, and is used for classification and search.

[0141] "Billing information" is data including the billing amount calculated based on the usage record of each company and its details, and is managed as a bill.

[0142] This invention relates to a generative AI platform for small and medium-sized enterprises, which supports their digitalization and productivity improvement by providing dedicated and shared environments for each company. This platform integrates a dedicated database, secure authentication management, confidential data processing, utilization of specialized knowledge, audio and video data management, and proper processing of billing information.

[0143] Setting up a dedicated environment

[0144] The server creates a dedicated database for each small or medium-sized business, allowing data to be kept separate and preventing it from being mixed with other companies' data. The database contains tables for sales, customers, invoices, etc., and can be managed exclusively by each company.

[0145] The server then initializes an administrator account and sends the credentials to the company, using the bcrypt library for password hashing. After receiving the administrator account, the company administrator can add employee accounts and set the necessary permissions after the first login.

[0146] Additionally, the server uses an ID management service such as Auth0 to securely manage authentication information and performs authentication using JWT tokens, implementing advanced security measures to prevent unauthorized access.

[0147] Data entry and management

[0148] Users log in to the company's dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form, and by clicking the "Save" button, the data is sent to the server.

[0149] The device sends the entered data to a company-specific API endpoint and makes a save request to the server. This data is sent using HTTPS, ensuring security. The server receives the data, validates it, and saves it in a MySQL (registered trademark) database. The user is notified when the save is successful.

[0150] Leveraging expertise in a common environment

[0151] Users access the common environment and enter their tax or legal questions into a text form, enter a specific question such as "How do I file my taxes the latest?", and click the "Submit" button.

[0152] The device sends the question to the common environment and generates a prompt to search for the relevant information in the knowledge base. The server retrieves the appropriate information from the knowledge base and returns it to the user, allowing the user to instantly access the latest legal and tax information.

[0153] Audio and video data management

[0154] Users can record their work details and know-how in audio and video formats. For example, they can record a video of the assembly procedure for a new product on their smartphone and upload it to the system.

[0155] The device sends the uploaded audio and video data to a company-specific storage (e.g., AWS S3). The server converts the audio to text using AWS Transcribe and stores it as metadata in DynamoDB. This metadata makes it easily searchable and browsable at a later date.

[0156] Manage your billing information

[0157] The server aggregates the usage data for each company monthly, calculates the billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator user.

[0158] The terminal then displays the billing information to the administrator and provides an online payment link, which uses the Stripe payment gateway and automatically updates records in the system once payment is completed. Admin users can view their invoices and quickly pay online.

[0159] Prompt Sentence Examples

[0160] 1. "How do I enter new sales data?"

[0161] 2. "How do I file my taxes the latest?"

[0162] By implementing these procedures, the present invention solves many of the problems faced by small and medium-sized enterprises, and improves business efficiency and productivity.

[0163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0164] Setting up a dedicated environment

[0165] Step 1:

[0166] The server creates a database for each small or medium-sized business. Specifically, it uses a MySQL database with a different schema for each company. For example, for Company A, it creates tables for sales, customers, invoices, etc.

[0167] Input: Small and Medium Enterprise Information

[0168] Output: Company-specific database

[0169] Step 2:

[0170] The server initializes an administrator account, hashes the credentials using the bcrypt library, and then emails the resulting credentials to the company.

[0171] Input: Administrator information

[0172] Output: Admin account with hashed credentials

[0173] Step 3:

[0174] The user (administrator) logs into the dedicated environment and adds accounts for other employees. The administrator sets the appropriate permissions for each account and grants the access rights required for work.

[0175] Input: Employee information

[0176] Output: Permissioned employee accounts

[0177] Step 4:

[0178] The server manages employee account authentication information, uses identity management services such as Auth0 to prevent unauthorized access, generates JWT tokens, and implements security measures.

[0179] Input: Employee Credentials

[0180] Output: Secure credentials

[0181] Data entry and management

[0182] Step 1:

[0183] Users log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form and then the user clicks the "Save" button.

[0184] Input: Sales data, customer information

[0185] Output: Form data

[0186] Step 2:

[0187] The device sends the data entered by the user in JSON format to the enterprise API endpoint and makes a save request to the server. This request is sent using HTTPS, ensuring security.

[0188] Input: Form Data

[0189] Output: API request

[0190] Step 3:

[0191] The server validates the data received from the terminal and saves it in the appropriate table (sales, customer, etc.) in the dedicated database. It notifies the terminal that saving is complete.

[0192] Input: API request data

[0193] Output: Database entry, save success notification

[0194] Leveraging expertise in a common environment

[0195] Step 1:

[0196] Users access the common environment, enter a question, such as "How do I file my taxes the latest?" into a text form, and click the "Submit" button.

[0197] Input: Question

[0198] Output: Question submitted

[0199] Step 2:

[0200] The terminal transmits the transmitted question to the common environment and generates a prompt to search for the relevant information within the knowledge base.

[0201] Input: Submitted question

[0202] Output: Search prompt

[0203] Step 3:

[0204] The server retrieves relevant information from the knowledge base, generates an answer, and sends it back to the device, providing detailed guidance based on tax or legal information, for example.

[0205] Input: Search prompt

[0206] Output: Answer information

[0207] Audio and video data management

[0208] Step 1:

[0209] Users record and record their work content and know-how in audio or video format and upload the files to the system.

[0210] Input: Audio files, video files

[0211] Output: Upload request

[0212] Step 2:

[0213] The device sends the uploaded audio and video data to the company's dedicated storage and makes a storage request to the server, for example, AWS S3, to store the data securely.

[0214] Input: Upload request

[0215] Output: Save request to storage

[0216] Step 3:

[0217] The server converts the audio to text using AWS Transcribe and stores the text data along with metadata in DynamoDB, which makes it easier to search and categorize later.

[0218] Input: Storage data

[0219] Output: Text data, metadata

[0220] Manage your billing information

[0221] Step 1:

[0222] The server aggregates each company's monthly usage data, calculates the appropriate billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator.

[0223] Input: Monthly usage record

[0224] Output: Invoice PDF

[0225] Step 2:

[0226] The terminal displays billing information to the administrator and provides an online payment link, using the Stripe payment gateway.

[0227] Input: Invoice PDF, administrator information

[0228] Output: Payment link

[0229] Step 3:

[0230] The admin user reviews the invoice and clicks on the provided link to complete the payment online. Once the payment is successful, the transaction record is automatically updated in the system.

[0231] Input: Payment information

[0232] Output: Updated transaction record

[0233] (Application example 1)

[0234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0235] In recent years, the slow pace of digitalization among small and medium-sized enterprises has become a problem. While there is a demand for improved productivity, technical resources are often lacking. In particular, for companies that use automated vehicles, managing and optimizing vehicle operation data requires a great deal of time and effort, which leads to a decline in business efficiency. In addition, it is difficult to safely manage confidential data and share specialized knowledge, and audio and video data are not being effectively utilized. There is a need for a system that can solve these issues and support the digitalization and productivity improvement of small and medium-sized enterprises.

[0236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0237] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for collecting operation data of automatically controlled vehicles and transmitting it to a company-dedicated server, and means for analyzing the operation data and optimizing operation plans in real time. This enables efficient management and optimization of operation data of automatically controlled vehicles, enabling digitalization and improved productivity for small and medium-sized enterprises.

[0238] A "database" is a system for systematically organizing, storing, and managing information.

[0239] "Authentication information" is information used to verify the authenticity of a user or system.

[0240] "Confidential data" refers to important data that could cause harm if leaked to the outside.

[0241] A "knowledge base" is a database that systematically compiles specific knowledge and information.

[0242] "Audio data" means data that is a digital representation of a recording of spoken words or other sounds.

[0243] "Moving image data" is digital data that expresses movement by playing back a series of still images at regular time intervals.

[0244] "Billing information" is information indicating the cost for providing a service or product.

[0245] An "automated vehicle" is a vehicle that can operate automatically without human operation.

[0246] "Operation data" refers to various data related to operation, such as the vehicle's position, speed, and condition.

[0247] A "server" is a computer system that provides services over a network.

[0248] "Real-time" refers to a state in which information and processing are carried out immediately without delay.

[0249] This invention is a system that collects and manages operational data from autonomous vehicles to support the digitalization and productivity improvement of small and medium-sized enterprises. Specifically, it creates a dedicated database for each company, securely manages authentication information and confidential data, and provides an environment where a common knowledge base can be easily accessed. The system also includes functions such as uploading and managing audio and video data, generating and managing billing information for each company, and collecting, analyzing, and optimizing operational data from autonomous vehicles.

[0250] System Configuration

[0251] 1. Creation and management of a dedicated database for each company

[0252] The server creates a dedicated database for each company and stores confidential data in that database. Each company's authentication information is managed in this database, and security measures are implemented to prevent unauthorized access. For example, an administrator completes the initial login and adds employee accounts, setting appropriate permissions.

[0253] 2. Access to a common, dedicated knowledge base

[0254] Users can use a common, dedicated knowledge base to ask tax and legal questions and get answers in real time, improving work efficiency. For example, by entering a question like "What is the latest tax filing method?" into the system, the latest information is retrieved from the common environment.

[0255] 3. Audio and video data management

[0256] Users record their work and know-how in audio or video format and upload it to the system. The server analyzes the audio data, converts it into text, and generates metadata, which makes it easy for other employees to search and refer to. For example, a video of the assembly procedure for a new product can be recorded and made available to each employee.

[0257] 4. Management of operational data for autonomous vehicles

[0258] Operational data from autonomous vehicles is sent in real time from the vehicle's terminal to a company-dedicated server. This data includes location information, battery level, and operating status. The server analyzes this data and optimizes operation plans in real time. For example, if the battery level is low, it will guide the vehicle to the nearest charging station.

[0259] 5. Generative AI Model and Prompts

[0260] Generative AI models are used to perform various data analyses and optimizations. For example, a prompt such as "Please tell me how to develop a system that manages autonomous vehicle operation data in real time and provides optimal operation plans" can be generated and input into the AI ​​model to achieve advanced analysis.

[0261] Hardware and Software Examples

[0262] Hardware: Servers, smartphones, autonomous vehicle terminals

[0263] Software: Python, requests library, enterprise-specific API endpoints

[0264] The above configuration and means will enable digitalization and productivity improvement in small and medium-sized enterprises.

[0265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0266] Step 1:

[0267] The user logs into the dedicated environment and inputs vehicle operation data. Specifically, the user enters data such as the vehicle's location, battery level, and operating status into an input form and clicks the "Submit" button. The input data also includes the vehicle's identification information.

[0268] Step 2:

[0269] The device sends the operational data entered by the user to a company-specific API endpoint, where it is packaged in JSON format and sent as an HTTP POST request to the API endpoint. The device processes the data and sends it to the server via a communication protocol.

[0270] Step 3:

[0271] The server analyzes the operational data received from the terminal and stores it in the company's dedicated database. The data is converted into the appropriate format and inserted into the specified table in the database. Once the data has been saved, the server notifies the terminal that the data has been saved successfully.

[0272] Step 4:

[0273] The server performs real-time analysis based on the stored operational data to optimize the operation plan. This analysis involves statistically processing multidimensional data such as location information, battery level, and operation status to select the optimal operation route and charging station. Generative AI models are utilized here.

[0274] Step 5:

[0275] The server sends the analysis results back to the device via a dedicated API endpoint. Specifically, it includes an optimized operation plan and instructions for when emergency charging is required. The device receives this analysis data and notifies the user.

[0276] Step 6:

[0277] The user checks the analysis results notified from the device and modifies the operation plan as necessary. For example, if an instruction to go to a charging station is issued, the user will act promptly in accordance with the instruction. The input in this step is the analysis results, and the output is the user's action.

[0278] The above processing steps seamlessly realize a series of steps including data collection, analysis, optimization, and notification, enabling small and medium-sized enterprises to digitalize and improve productivity.

[0279] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0280] This invention combines a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions to provide a more personalized user experience and promote business efficiency.

[0281] Overall system overview

[0282] The system provides each small and medium-sized enterprise with a dedicated database, safely managing each company's confidential data and creating an environment where they can utilize shared expertise. It also includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[0283] Setting up a dedicated environment

[0284] 1. Server:

[0285] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[0286] Initialize the administrator account and provide credentials to the company.

[0287] 2. User (Administrator):

[0288] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[0289] 3. Server:

[0290] Manage employee account credentials and implement security measures to prevent unauthorized access.

[0291] Data entry and management

[0292] 1. User:

[0293] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[0294] Use a specific data entry form and click the "Save" button.

[0295] 2. Terminal:

[0296] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[0297] 3. Server:

[0298] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[0299] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[0300] Leveraging expertise in a common environment

[0301] 1. User:

[0302] Enter your tax and legal questions in a common environment and click the "Submit" button.

[0303] 2. Terminal:

[0304] The user's question is sent to a common environment and the relevant information is searched for.

[0305] 3. Server:

[0306] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0307] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[0308] Audio and video data management

[0309] 1. User:

[0310] Record and upload business content and know-how in audio or video format.

[0311] 2. Terminal:

[0312] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[0313] 3. Server:

[0314] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0315] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[0316] Manage your billing information

[0317] 1. Server:

[0318] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[0319] 2. Terminal:

[0320] Display billing information to administrators and provide payment links.

[0321] 3. Admin user:

[0322] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[0323] Utilizing the Emotion Engine

[0324] 1. User:

[0325] Interact with the system through voice and text.

[0326] 2. Emotion Engine:

[0327] Analyzes the user's voice and text to recognize emotions.

[0328] 3. Server:

[0329] It tailors its response based on the perceived emotion and provides personalized feedback to the user.

[0330] 4. Terminal:

[0331] The tailored response is displayed to the user, improving the user's experience.

[0332] Example: If a user gives a speech input that indicates frustration to the system, the emotion engine will recognize that emotion and the system will provide a more supportive response.

[0333] Accumulation and use of emotional data

[0334] 1. Emotion Engine:

[0335] Emotional data for each user is accumulated and saved as a history.

[0336] 2. Server:

[0337] The accumulated emotional data is analyzed to extract trends and patterns for individual users.

[0338] 3. Terminal:

[0339] The analysis results are used to further personalize the user experience and provide adaptive feedback.

[0340] Example: If a user regularly expresses negative sentiment towards the same question, the system can take action to improve the answer to that question and increase user satisfaction.

[0341] In this way, the present invention combines data management for each SME, utilization of common expertise, and an emotion engine to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[0342] The processing flow will be explained below.

[0343] Setting up a dedicated environment

[0344] Step 1:

[0345] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[0346] Step 2:

[0347] The server initializes the administrator account and provides the credentials to the company.

[0348] Step 3:

[0349] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[0350] Step 4:

[0351] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[0352] Data entry and management

[0353] Step 1:

[0354] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[0355] Step 2:

[0356] A user uses a particular data entry form and clicks the "Save" button.

[0357] Step 3:

[0358] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[0359] Step 4:

[0360] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[0361] Leveraging expertise in a common environment

[0362] Step 1:

[0363] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[0364] Step 2:

[0365] The terminal transmits the user's question to the common environment and searches for relevant information.

[0366] Step 3:

[0367] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0368] Audio and video data management

[0369] Step 1:

[0370] Users record and upload their business operations and know-how in audio and video formats.

[0371] Step 2:

[0372] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[0373] Step 3:

[0374] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0375] Manage your billing information

[0376] Step 1:

[0377] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[0378] Step 2:

[0379] The server generates an invoice and notifies the administrative user.

[0380] Step 3:

[0381] The terminal displays billing information to the administrator and provides a payment link.

[0382] Step 4:

[0383] Admin user reviews invoices and makes payments online.

[0384] Step 5:

[0385] The server updates the records in the system when the payment is completed.

[0386] Utilizing the Emotion Engine

[0387] Step 1:

[0388] The user interacts with the system through voice and text.

[0389] Step 2:

[0390] The emotion engine analyzes the user's voice and text to recognize emotions.

[0391] Step 3:

[0392] The server tailors the response based on the recognized emotion and provides personalized feedback to the user.

[0393] Step 4:

[0394] The terminal displays the adjusted response to the user, improving the user experience.

[0395] Accumulation and use of emotional data

[0396] Step 1:

[0397] The emotion engine accumulates each user's emotion data and saves it as a history.

[0398] Step 2:

[0399] The server analyzes the accumulated emotional data and extracts trends and patterns for individual users.

[0400] Step 3:

[0401] The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[0402] Example 2

[0403] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0404] Small and medium-sized enterprises (SMEs) have a wide range of tasks, including data management, utilizing specialized knowledge, and managing billing information, making it difficult to perform these tasks efficiently. Furthermore, they lack a system that can properly recognize employee emotions and provide personalized feedback. Therefore, there is a need for a system that can improve both work efficiency and employee satisfaction.

[0405] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0406] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for saving and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for recognizing user emotions, accumulating and analyzing emotion data, and means for adjusting responses and providing personalized feedback based on the recognized emotions, thereby enabling small and medium-sized enterprises to efficiently manage data and utilize knowledge, and to provide personalized feedback according to user emotions.

[0407] "Company-specific database" refers to an individual database set up by each company to securely store and manage its own data.

[0408] "Authentication Information" means a username, password, or other authentication means to grant access to a System.

[0409] "Confidential data" refers to important information held within a company, such as sales data, customer information, and business know-how.

[0410] A "specialized knowledge base" refers to a database that compiles information and guidelines on a specific field.

[0411] "Means for uploading and managing audio and video" refers to the functionality that allows users to upload audio and video data to the system and store and manage it.

[0412] "Billing Information" refers to the fees calculated based on each company's service usage record and information related to the payment thereof.

[0413] "Means for recognizing emotions and storing and analyzing emotional data" refers to the function of analyzing emotions from the user's voice and text, and collecting, saving, and analyzing that data.

[0414] "Means for adjusting responses and providing personalized feedback based on recognized emotions" refers to the ability of the system to dynamically generate responses and provide individualized feedback based on the user's emotions.

[0415] This invention is a system that provides a more personalized user experience and promotes business efficiency by combining a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions. This system includes the following elements.

[0416] Overall system overview

[0417] The system provides each small and medium-sized enterprise with a dedicated database to securely manage confidential data for each company. It also accesses a common dedicated knowledge base and includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[0418] Setting up a dedicated environment

[0419] 1. Server:

[0420] We create a dedicated database for each small and medium-sized business and provide an environment where corporate data can be safely stored, using database systems such as MySQL and PostgreSQL.

[0421] Initialize an administrator account and provide credentials to the company, which are managed using a framework such as Django or Ruby on Rails.

[0422] 2. User (Administrator):

[0423] After logging in for the first time, you can add accounts for other employees and set the appropriate permissions for each account. Administrative functions are performed through the web interface.

[0424] 3. Server:

[0425] Employee account authentication information is managed using security services such as Auth0 and Firebase, and security measures are implemented to prevent unauthorized access.

[0426] Data entry and management

[0427] 1. User:

[0428] Log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a data entry form built on a front-end framework (e.g., React, Vue.js).

[0429] 2. Terminal:

[0430] The entered data is sent to a company-specific RESTful API endpoint, and a save request is sent to the server. The communication protocol is HTTP.

[0431] 3. Server:

[0432] Save the data received from the device in a company-specific database and notify the device when the save is complete. Use a backend framework (e.g., Express.js, Spring Boot).

[0433] As a specific example of how it works, users can input daily sales data and the system will save this data in real time, allowing them to instantly grasp sales trends and helping to improve business operations.

[0434] Leveraging expertise in a common environment

[0435] 1. User:

[0436] Enter your tax and legal questions in a common environment and click the "Submit" button.

[0437] 2. Terminal:

[0438] The user's question is sent to a common environment and the relevant information is searched for.

[0439] 3. Server:

[0440] It retrieves relevant information from a dedicated knowledge base in the common environment and sends it back to the device, using a wiki or other information resource.

[0441] As a specific example of how it works, when a user asks "How to file tax returns the latest?", the system retrieves the latest information from the common environment and provides detailed guidance.

[0442] Audio and video data management

[0443] 1. User:

[0444] Record and upload business content and know-how in audio or video format.

[0445] 2. Terminal:

[0446] The recording and video data is sent to the company's dedicated cloud storage (e.g., Amazon S3) and a storage request is sent to the server.

[0447] 3. Server:

[0448] The uploaded audio data is converted into text using the Google(R) Cloud Speech-to-Text API, metadata is generated, and the data is indexed in ElasticSearch(R).

[0449] As a specific example of how it works, a user can record a video of the assembly procedure for a new product, and the system will analyze the video, making it easy for other employees to search and refer to.

[0450] Manage your billing information

[0451] 1. Server:

[0452] Calculate the billing amount for each company based on monthly usage. Generate an invoice using Stripe API and notify the administrator user.

[0453] 2. Terminal:

[0454] Display billing information to administrators and provide payment links.

[0455] 3. Admin user:

[0456] View your invoice and pay online. Once payment is complete, your records will be automatically updated in the system.

[0457] Utilizing the Emotion Engine and Response Adjustment

[0458] 1. User:

[0459] Interact with the system via voice and text.

[0460] 2. Emotion Engine:

[0461] It uses IBM Watson's (registered trademark) Natural Language Understanding API to analyze the user's voice and text and recognize emotions.

[0462] 3. Server:

[0463] It tailors responses based on perceived emotions and provides personalized feedback using generative AI models.

[0464] 4. Terminal:

[0465] The tailored response is displayed to the user, improving the user's experience.

[0466] As a specific example of how this works, if a user makes a speech input to the system that indicates frustration, the emotion engine will recognize the emotion and the system will generate and display a more supportive and understanding response.

[0467] Accumulation and analysis of emotional data

[0468] 1. Emotion Engine:

[0469] Emotion data for each user is periodically accumulated.

[0470] 2. Server:

[0471] The accumulated emotional data is analyzed using machine learning models (e.g., TENSORFLOW (registered trademark), PyTorch) to extract trends and patterns for individual users.

[0472] 3. Terminal:

[0473] Based on the analysis results, the user experience can be further personalized and adaptive feedback can be provided.

[0474] As a specific example of how it works, if a user repeatedly has a negative reaction to a particular question, that emotional data is accumulated and analyzed, and the system improves the answer to that question.

[0475] An example prompt might be, "What is the latest information on tax filing? I currently run a small business and need advice on filing my 2023 taxes."

[0476] As described above, the present invention combines data management for each SME with the utilization of common expertise, as well as an emotion engine, to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[0477] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0478] Step 1: User authentication

[0479] Input: The user enters their username and password on the login screen.

[0480] Processing: The device sends the input information to the Auth0 API. Auth0 verifies the authentication information and generates an authentication token.

[0481] Output: The server receives the authentication token and returns it to the device indicating that the user has been authenticated.

[0482] What it does: When a user enters their credentials on the login screen, the system verifies them through Auth0 and redirects them to the dashboard if authentication is successful.

[0483] Step 2: Enter and save data

[0484] Input: Users enter sensitive data, such as sales data or customer information, into data entry forms.

[0485] Processing: The device sends the input data to a company-specific server using a RESTful API, which stores the data in a MySQL database.

[0486] Output: The server notifies the device that the save is complete.

[0487] Specific behavior: When a user enters sales data into a React-based data entry form and clicks the "Save" button, the data is sent to the server and saved in the database.

[0488] Step 3: Find expertise

[0489] Input: User enters tax or legal question in a common environment and clicks "Submit" button.

[0490] Processing: The device sends the query to the common environment search API to search for related information.

[0491] Output: The server retrieves the relevant information from the knowledge base and returns the search results to the terminal.

[0492] What it does: When a user types "latest tax filing method" into the search bar and hits the submit button, the system displays the latest relevant tax guidelines.

[0493] Step 4: Managing audio and video data

[0494] Input: Users record and record their work content and know-how in audio or video format and upload it to the system.

[0495] Processing: The device sends the audio and video data to the company's dedicated cloud storage and sends a storage request to the server. The server analyzes the audio data and converts it into text.

[0496] Output: The server generates the parsed text and metadata and indexes it into Elasticsearch.

[0497] Specific operation: When a user records a video of their business know-how and uploads it to the system, the audio is automatically converted into text and saved in a searchable format.

[0498] Step 5: Manage your billing information

[0499] Input: The server generates billing information based on monthly usage data.

[0500] Processing: The server uses the Stripe API to generate invoices for each company and notify the admin user.

[0501] Output: The terminal displays the billing information to the administrator and provides a payment link.

[0502] Specific operation: At the end of each month, the system will compile usage data, automatically generate an invoice through Stripe, and notify the administrator user. The administrator can then log in, check the invoice, and pay online.

[0503] Step 6: Leverage your emotional engine and adjust your response

[0504] Input: The user interacts with the system through voice and text.

[0505] Processing: The emotion engine uses IBM Watson APIs to analyze the user's emotions. The server tailors responses based on the recognized emotions and provides personalized feedback using generative AI models.

[0506] Output: The device displays the adjusted response to the user, improving the user's experience.

[0507] How it works: When a user inputs speech that expresses dissatisfaction with the system, the emotion engine recognizes the emotion and the system provides a supportive response.

[0508] Step 7: Collect and analyze emotional data

[0509] Input: The emotion engine periodically collects emotion data for each user.

[0510] Processing: The server analyzes the accumulated emotional data using machine learning models to extract trends and patterns of the user's emotions.

[0511] Output: The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[0512] What it does: If a user expresses negative sentiment towards a particular question, that data is stored in the system to improve future answers.

[0513] (Application example 2)

[0514] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0515] The purpose of this invention is to improve business efficiency and productivity in small and medium-sized enterprises. Previous systems did not adequately utilize company-specific data management and specialized knowledge, making it difficult to optimize business processes. Furthermore, there was a lack of personalized responses and support that took employee emotions into consideration, leaving the issue of improving the working environment unresolved.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0517] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for interpreting voice instructions and dynamically changing work plans, and means for recognizing user emotions using an emotion engine and adjusting responses, thereby enabling more efficient data management, optimization of business processes, and personalized support based on employee emotions.

[0518] A "company-specific database" is a database in which data is securely stored and managed in a format that each company can use independently.

[0519] "Authentication information" is identification information required when a user accesses a system, and is information used to ensure security.

[0520] "Confidential data" refers to important information held by a company that must not be leaked to the outside.

[0521] A "dedicated knowledge base" is a database in which specialized knowledge and information are accumulated and managed, and can be shared when necessary.

[0522] "Audio and video uploading and management" refers to the process by which users provide audio and video data to the system, which then stores and organizes that data.

[0523] "Billing information" refers to information regarding fees and payments calculated based on usage history.

[0524] "Means for interpreting voice instructions and dynamically changing work plans" refers to a function that understands instructions provided by voice and automatically adjusts the schedule and content of work.

[0525] An "emotion engine" is a system that analyzes the user's emotions and generates and adjusts appropriate responses based on those emotions.

[0526] The present invention aims to improve business efficiency and productivity in small and medium-sized enterprises. Specific embodiments will be described below.

[0527] Overall system configuration

[0528] The system consists of the following main elements:

[0529] Server Roles

[0530] The server creates a dedicated database for each company, manages authentication information for each company, stores and manages confidential data, accesses a common dedicated knowledge base, uploads and manages audio and video data, and generates and manages billing information for each company.

[0531] Device Role

[0532] The device provides an interface for users to issue voice commands, interprets the voice commands, dynamically changes the task plan, and uses an emotion engine to recognize the user's emotions and adjust responses accordingly.

[0533] User Roles

[0534] Users interact with the system by giving instructions to it through voice or text, for example, they can ask the system to explain a specific work procedure.

[0535] Main processing flow

[0536] Speech recognition processing

[0537] The device uses voice recognition technology to analyze user instructions, using a microphone and a Raspberry Pi to convert voice data into text using the SpeechRecognition library.

[0538] Emotion recognition processing

[0539] The emotion engine analyzes emotions from the user's voice or text using an external emotion recognition API, for example, to determine whether the user is frustrated based on the content of their voice instructions.

[0540] Data storage and sharing

[0541] The audio and video data collected from users is stored on a cloud server (e.g., AWS, Google Cloud). PostgreSQL or MongoDB is used as the database. The data is managed in a way that makes it properly categorized and searchable.

[0542] Exception handling assistance

[0543] If the emotion engine detects negative emotions from the user, the system will provide guidance for immediate action and automatically initiate assistance through technical staff if support is needed.

[0544] Specific examples

[0545] For example, in a scenario where a worker is explaining the production procedure for a new part in a factory, the worker might give the robot a voice command such as "Explain the production procedure for the new part." In this case:

[0546] 1. The device analyzes the voice commands and converts them into text.

[0547] 2. The emotion engine recognizes the user's emotions and, if it determines that the worker is feeling frustrated, it presents detailed instructions in video format.

[0548] 3. Detailed visual guides are provided to help workers understand better.

[0549] Prompt Sentence Examples

[0550] Here are some example prompts to input to a generative AI model:

[0551] text

[0552] A worker commands the robot to "explain the production procedure for a new part." The emotion engine recognizes that the worker is expressing frustration. In this case, the robot displays a detailed video of the procedure to help the worker understand.

[0553] In this way, the system provided by the present invention not only achieves data management and business process optimization for small and medium-sized enterprises, but also provides flexible support according to the emotional state of employees.

[0554] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0555] Step 1:

[0556] The user gives a voice command such as "Explain the production procedure for the new part."

[0557] Input: A spoken command from the user.

[0558] Output: Audio data.

[0559] Specific action: The user speaks into the device's microphone.

[0560] Step 2:

[0561] The terminal uses voice recognition software to convert the voice data into text.

[0562] Input: Audio data.

[0563] Output: Textual instructions.

[0564] Specific operation: Uses the SpeechRecognition library to convert voice data into text. Uses devices such as Raspberry Pi.

[0565] Step 3:

[0566] The device sends the converted text to the emotion engine's API to analyze the user's emotions.

[0567] Input: Instructions in text format.

[0568] Output: Sentiment analysis results.

[0569] Specific operation: The device sends text data to the emotion recognition API, and the emotion engine analyzes the emotion.

[0570] Step 4:

[0571] If the emotion engine detects frustration, the device will display supportive instructions.

[0572] Input: Sentiment analysis results.

[0573] Output: Instructions and procedures for support.

[0574] Specific operation: Based on the analysis results, the device displays an easy-to-understand visual guide to the user.

[0575] Step 5:

[0576] The server stores the emotion engine's analysis results and the contents of the voice instructions as a log.

[0577] Input: Sentiment analysis results and instructions in text format.

[0578] Output: Log data.

[0579] Specific operation: Works with a cloud server (e.g. AWS, Google Cloud) to store results in a database.

[0580] Step 6:

[0581] The user follows the provided visual guide as they work.

[0582] Enter: a visual guide.

[0583] Output: User's work progress.

[0584] Specific operation: The user learns and executes the production procedure for a new part by following the video and text guide displayed on the device screen.

[0585] By providing a detailed explanation of the processing performed at each step and clearly showing the overall flow of the system, the quality of the user experience can be improved and business efficiency for small and medium-sized enterprises can be improved.

[0586] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0587] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0588] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0589] [Second embodiment]

[0590] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0591] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0592] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0593] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0594] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0595] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0596] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0597] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0598] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0599] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0600] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0601] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0602] This invention is a generative AI platform for small and medium-sized enterprises, which supports the digitalization and productivity improvement of small and medium-sized enterprises by providing dedicated environments for each company and a common environment.

[0603] Overall system overview

[0604] The system provides each small and medium-sized enterprise with a dedicated database, safely manages confidential data for each company, and creates an environment where common expertise can be utilized. It also includes a variety of functions, such as managing audio and video data, sharing business know-how, and processing billing information.

[0605] Setting up a dedicated environment

[0606] 1. Server:

[0607] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[0608] Initialize the administrator account and provide credentials to the company.

[0609] 2. User (Administrator):

[0610] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[0611] 3. Server:

[0612] Manage employee account credentials and implement security measures to prevent unauthorized access.

[0613] Data entry and management

[0614] 1. User:

[0615] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[0616] Use a specific data entry form and click the "Save" button.

[0617] 2. Terminal:

[0618] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[0619] 3. Server:

[0620] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[0621] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[0622] Leveraging expertise in a common environment

[0623] 1. User:

[0624] Enter your tax and legal questions in a common environment and click the "Submit" button.

[0625] 2. Terminal:

[0626] The user's question is sent to a common environment and the relevant information is searched for.

[0627] 3. Server:

[0628] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0629] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[0630] Audio and video data management

[0631] 1. User:

[0632] Record and upload business content and know-how in audio or video format.

[0633] 2. Terminal:

[0634] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[0635] 3. Server:

[0636] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0637] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[0638] Manage your billing information

[0639] 1. Server:

[0640] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[0641] 2. Terminal:

[0642] Display billing information to administrators and provide payment links.

[0643] 3. Admin user:

[0644] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[0645] Example: At the end of each month, the server aggregates the usage history and generates the appropriate invoice, which the administrator can then view and easily complete the payment.

[0646] In this way, the present invention combines data management for each small and medium-sized enterprise with the utilization of common expertise to achieve efficient business operations and accurate billing processing using voice and video, thereby resolving many of the challenges faced by small and medium-sized enterprises and improving productivity.

[0647] The processing flow will be explained below.

[0648] Setting up a dedicated environment

[0649] Step 1:

[0650] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[0651] Step 2:

[0652] The server initializes the administrator account and provides the credentials to the company.

[0653] Step 3:

[0654] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[0655] Step 4:

[0656] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[0657] Data entry and management

[0658] Step 1:

[0659] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[0660] Step 2:

[0661] A user uses a particular data entry form and clicks the "Save" button.

[0662] Step 3:

[0663] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[0664] Step 4:

[0665] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[0666] Leveraging expertise in a common environment

[0667] Step 1:

[0668] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[0669] Step 2:

[0670] The terminal transmits the user's question to the common environment and searches for relevant information.

[0671] Step 3:

[0672] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0673] Audio and video data management

[0674] Step 1:

[0675] Users record and upload their business operations and know-how in audio and video formats.

[0676] Step 2:

[0677] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[0678] Step 3:

[0679] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0680] Manage your billing information

[0681] Step 1:

[0682] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[0683] Step 2:

[0684] The server generates an invoice and notifies the administrative user.

[0685] Step 3:

[0686] The terminal displays billing information to the administrator and provides a payment link.

[0687] Step 4:

[0688] Admin user reviews invoices and makes payments online.

[0689] Step 5:

[0690] The server updates the records in the system when the payment is completed.

[0691] Example 1

[0692] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0693] Currently, many small and medium-sized enterprises have not made much progress in digitalization, and are facing challenges in business efficiency and productivity. Furthermore, it is difficult to utilize the different data management and expertise that each company has, and in particular, the management of audio and video data and proper billing processing are insufficient. As a result, they are unable to obtain real-time information or appropriate advice for daily operations and management decisions, which is hindering productivity improvement.

[0694] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0695] In this invention, the server includes a means for generating a database for storing data specific to each company, a means for managing authentication information for each company, a means for inputting, saving, and managing confidential data for each company, a means for accessing a common knowledge base and providing related information, a means for uploading, managing, and analyzing audio and video data, and a means for generating and managing billing information based on each company's usage history. This allows small and medium-sized enterprises to promote digitalization, obtain information in real time, and receive appropriate advice. It also improves business efficiency and accurate billing processing, significantly increasing productivity.

[0696] A "database" is a system for managing, storing, and efficiently manipulating a collection of data created specifically for each company.

[0697] "Authentication information" refers to information required for a user to access a system, and is primarily composed of a username, password, security token, and the like.

[0698] "Confidential data" refers to data that should only be used within a specific company, such as a company's sales data or customer information, and must be managed with high security.

[0699] A "knowledge base" is a database in which various specialized knowledge and information are organized and stored, and is used to provide appropriate answers to user questions.

[0700] "Audio data" is a recording of a user's work content or know-how, and is mainly saved in the form of an audio file.

[0701] "Video data" refers to work content or know-how recorded by a user, and is mainly saved in the form of a video file.

[0702] "Uploading" is the process by which a user sends data from their device to a server and stores it.

[0703] "Analysis" is the process of analyzing uploaded audio or video, converting the content into text, and generating metadata.

[0704] "Metadata" is auxiliary information added to facilitate data management and search, and is used for classification and search.

[0705] "Billing information" is data including the billing amount calculated based on the usage record of each company and its details, and is managed as a bill.

[0706] This invention relates to a generative AI platform for small and medium-sized enterprises, which supports their digitalization and productivity improvement by providing dedicated and shared environments for each company. This platform integrates a dedicated database, secure authentication management, confidential data processing, utilization of specialized knowledge, audio and video data management, and proper processing of billing information.

[0707] Setting up a dedicated environment

[0708] The server creates a dedicated database for each small or medium-sized business, allowing data to be kept separate and preventing it from being mixed with other companies' data. The database contains tables for sales, customers, invoices, etc., and can be managed exclusively by each company.

[0709] The server then initializes an administrator account and sends the credentials to the company, using the bcrypt library for password hashing. After receiving the administrator account, the company administrator can add employee accounts and set the necessary permissions after the first login.

[0710] Additionally, the server uses an ID management service such as Auth0 to securely manage authentication information and performs authentication using JWT tokens, implementing advanced security measures to prevent unauthorized access.

[0711] Data entry and management

[0712] Users log in to the company's dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form, and by clicking the "Save" button, the data is sent to the server.

[0713] The device sends the entered data to a company-specific API endpoint and makes a save request to the server. This data is sent using HTTPS, ensuring security. The server receives the data, validates it, and saves it in a MySQL database. The user is notified when the save is successful.

[0714] Leveraging expertise in a common environment

[0715] Users access the common environment and enter their tax or legal questions into a text form, enter a specific question such as "How do I file my taxes the latest?", and click the "Submit" button.

[0716] The device sends the question to the common environment and generates a prompt to search for the relevant information in the knowledge base. The server retrieves the appropriate information from the knowledge base and returns it to the user, allowing the user to instantly access the latest legal and tax information.

[0717] Audio and video data management

[0718] Users can record their work details and know-how in audio and video formats. For example, they can record a video of the assembly procedure for a new product on their smartphone and upload it to the system.

[0719] The device sends the uploaded audio and video data to a company-specific storage (e.g., AWS S3), where the server converts the audio to text using AWS Transcribe and stores it as metadata in DynamoDB, making it easily searchable and browsable later.

[0720] Manage your billing information

[0721] The server aggregates the usage data for each company monthly, calculates the billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator user.

[0722] The terminal then displays the billing information to the administrator and provides an online payment link, which uses the Stripe payment gateway and automatically updates records in the system once payment is completed. Admin users can view their invoices and quickly pay online.

[0723] Prompt Sentence Examples

[0724] 1. "How do I enter new sales data?"

[0725] 2. "How do I file my taxes the latest?"

[0726] By implementing these procedures, the present invention solves many of the problems faced by small and medium-sized enterprises, and improves business efficiency and productivity.

[0727] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0728] Setting up a dedicated environment

[0729] Step 1:

[0730] The server creates a database for each small or medium-sized business. Specifically, it uses a MySQL database with a different schema for each company. For example, for Company A, it creates tables for sales, customers, invoices, etc.

[0731] Input: Small and Medium Enterprise Information

[0732] Output: Company-specific database

[0733] Step 2:

[0734] The server initializes an administrator account, hashes the credentials using the bcrypt library, and then emails the resulting credentials to the company.

[0735] Input: Administrator information

[0736] Output: Admin account with hashed credentials

[0737] Step 3:

[0738] The user (administrator) logs into the dedicated environment and adds accounts for other employees. The administrator sets the appropriate permissions for each account and grants the access rights required for work.

[0739] Input: Employee information

[0740] Output: Permissioned employee accounts

[0741] Step 4:

[0742] The server manages employee account authentication information, uses identity management services such as Auth0 to prevent unauthorized access, generates JWT tokens, and implements security measures.

[0743] Input: Employee Credentials

[0744] Output: Secure credentials

[0745] Data entry and management

[0746] Step 1:

[0747] Users log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form and then the user clicks the "Save" button.

[0748] Input: Sales data, customer information

[0749] Output: Form data

[0750] Step 2:

[0751] The device sends the data entered by the user in JSON format to the enterprise API endpoint and makes a save request to the server. This request is sent using HTTPS, ensuring security.

[0752] Input: Form Data

[0753] Output: API request

[0754] Step 3:

[0755] The server validates the data received from the terminal and saves it in the appropriate table (sales, customer, etc.) in the dedicated database. It notifies the terminal that saving is complete.

[0756] Input: API request data

[0757] Output: Database entry, save success notification

[0758] Leveraging expertise in a common environment

[0759] Step 1:

[0760] Users access the common environment, enter a question, such as "How do I file my taxes the latest?" into a text form, and click the "Submit" button.

[0761] Input: Question

[0762] Output: Question submitted

[0763] Step 2:

[0764] The terminal transmits the transmitted question to the common environment and generates a prompt to search for the relevant information within the knowledge base.

[0765] Input: Submitted question

[0766] Output: Search prompt

[0767] Step 3:

[0768] The server retrieves relevant information from the knowledge base, generates an answer, and sends it back to the device, providing detailed guidance based on tax or legal information, for example.

[0769] Input: Search prompt

[0770] Output: Answer information

[0771] Audio and video data management

[0772] Step 1:

[0773] Users record and record their work content and know-how in audio or video format and upload the files to the system.

[0774] Input: Audio files, video files

[0775] Output: Upload request

[0776] Step 2:

[0777] The device sends the uploaded audio and video data to the company's dedicated storage and makes a storage request to the server, for example, AWS S3, to store the data securely.

[0778] Input: Upload request

[0779] Output: Save request to storage

[0780] Step 3:

[0781] The server converts the audio to text using AWS Transcribe and stores the text data along with metadata in DynamoDB, which makes it easier to search and categorize later.

[0782] Input: Storage data

[0783] Output: Text data, metadata

[0784] Manage your billing information

[0785] Step 1:

[0786] The server aggregates each company's monthly usage data, calculates the appropriate billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator.

[0787] Input: Monthly usage record

[0788] Output: Invoice PDF

[0789] Step 2:

[0790] The terminal displays billing information to the administrator and provides an online payment link, using the Stripe payment gateway.

[0791] Input: Invoice PDF, administrator information

[0792] Output: Payment link

[0793] Step 3:

[0794] The admin user reviews the invoice and clicks on the provided link to complete the payment online. Once the payment is successful, the transaction record is automatically updated in the system.

[0795] Input: Payment information

[0796] Output: Updated transaction record

[0797] (Application example 1)

[0798] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] In recent years, the slow pace of digitalization among small and medium-sized enterprises has become a problem. While there is a demand for improved productivity, technical resources are often lacking. In particular, for companies that use automated vehicles, managing and optimizing vehicle operation data requires a great deal of time and effort, which leads to a decline in business efficiency. In addition, it is difficult to safely manage confidential data and share specialized knowledge, and audio and video data are not being effectively utilized. There is a need for a system that can solve these issues and support the digitalization and productivity improvement of small and medium-sized enterprises.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0801] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for collecting operation data of automatically controlled vehicles and transmitting it to a company-dedicated server, and means for analyzing the operation data and optimizing operation plans in real time. This enables efficient management and optimization of operation data of automatically controlled vehicles, enabling digitalization and improved productivity for small and medium-sized enterprises.

[0802] A "database" is a system for systematically organizing, storing, and managing information.

[0803] "Authentication information" is information used to verify the authenticity of a user or system.

[0804] "Confidential data" refers to important data that could cause harm if leaked to the outside.

[0805] A "knowledge base" is a database that systematically compiles specific knowledge and information.

[0806] "Audio data" means data that is a digital representation of a recording of spoken words or other sounds.

[0807] "Moving image data" is digital data that expresses movement by playing back a series of still images at regular time intervals.

[0808] "Billing information" is information indicating the cost for providing a service or product.

[0809] An "automated vehicle" is a vehicle that can operate automatically without human operation.

[0810] "Operation data" refers to various data related to operation, such as the vehicle's position, speed, and condition.

[0811] A "server" is a computer system that provides services over a network.

[0812] "Real-time" refers to a state in which information and processing are carried out immediately without delay.

[0813] This invention is a system that collects and manages operational data from autonomous vehicles to support the digitalization and productivity improvement of small and medium-sized enterprises. Specifically, it creates a dedicated database for each company, securely manages authentication information and confidential data, and provides an environment where a common knowledge base can be easily accessed. The system also includes functions such as uploading and managing audio and video data, generating and managing billing information for each company, and collecting, analyzing, and optimizing operational data from autonomous vehicles.

[0814] System Configuration

[0815] 1. Creation and management of a dedicated database for each company

[0816] The server creates a dedicated database for each company and stores confidential data in that database. Each company's authentication information is managed in this database, and security measures are implemented to prevent unauthorized access. For example, an administrator completes the initial login and adds employee accounts, setting appropriate permissions.

[0817] 2. Access to a common, dedicated knowledge base

[0818] Users can use a common, dedicated knowledge base to ask tax and legal questions and get answers in real time, improving work efficiency. For example, by entering a question like "What is the latest tax filing method?" into the system, the latest information is retrieved from the common environment.

[0819] 3. Audio and video data management

[0820] Users record their work and know-how in audio or video format and upload it to the system. The server analyzes the audio data, converts it into text, and generates metadata, which makes it easy for other employees to search and refer to. For example, a video of the assembly procedure for a new product can be recorded and made available to each employee.

[0821] 4. Management of operational data for autonomous vehicles

[0822] Operational data from autonomous vehicles is sent in real time from the vehicle's terminal to a company-dedicated server. This data includes location information, battery level, and operating status. The server analyzes this data and optimizes operation plans in real time. For example, if the battery level is low, it will guide the vehicle to the nearest charging station.

[0823] 5. Generative AI Model and Prompts

[0824] Generative AI models are used to perform various data analyses and optimizations. For example, a prompt such as "Please tell me how to develop a system that manages autonomous vehicle operation data in real time and provides optimal operation plans" can be generated and input into the AI ​​model to achieve advanced analysis.

[0825] Hardware and Software Examples

[0826] Hardware: Servers, smartphones, autonomous vehicle terminals

[0827] Software: Python, requests library, enterprise-specific API endpoints

[0828] The above configuration and means will enable digitalization and productivity improvement in small and medium-sized enterprises.

[0829] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0830] Step 1:

[0831] The user logs into the dedicated environment and inputs vehicle operation data. Specifically, the user enters data such as the vehicle's location, battery level, and operating status into an input form and clicks the "Submit" button. The input data also includes the vehicle's identification information.

[0832] Step 2:

[0833] The device sends the operational data entered by the user to a company-specific API endpoint, where it is packaged in JSON format and sent as an HTTP POST request to the API endpoint. The device processes the data and sends it to the server via a communication protocol.

[0834] Step 3:

[0835] The server analyzes the operational data received from the terminal and stores it in the company's dedicated database. The data is converted into the appropriate format and inserted into the specified table in the database. Once the data has been saved, the server notifies the terminal that the data has been saved successfully.

[0836] Step 4:

[0837] The server performs real-time analysis based on the stored operational data to optimize the operation plan. This analysis involves statistically processing multidimensional data such as location information, battery level, and operation status to select the optimal operation route and charging station. Generative AI models are utilized here.

[0838] Step 5:

[0839] The server sends the analysis results back to the device via a dedicated API endpoint. Specifically, it includes an optimized operation plan and instructions for when emergency charging is required. The device receives this analysis data and notifies the user.

[0840] Step 6:

[0841] The user checks the analysis results notified from the device and modifies the operation plan as necessary. For example, if an instruction to go to a charging station is issued, the user will act promptly in accordance with the instruction. The input in this step is the analysis results, and the output is the user's action.

[0842] The above processing steps seamlessly realize a series of steps including data collection, analysis, optimization, and notification, enabling small and medium-sized enterprises to digitalize and improve productivity.

[0843] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0844] This invention combines a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions to provide a more personalized user experience and promote business efficiency.

[0845] Overall system overview

[0846] The system provides each small and medium-sized enterprise with a dedicated database, safely managing each company's confidential data and creating an environment where they can utilize shared expertise. It also includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[0847] Setting up a dedicated environment

[0848] 1. Server:

[0849] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[0850] Initialize the administrator account and provide credentials to the company.

[0851] 2. User (Administrator):

[0852] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[0853] 3. Server:

[0854] Manage employee account credentials and implement security measures to prevent unauthorized access.

[0855] Data entry and management

[0856] 1. User:

[0857] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[0858] Use a specific data entry form and click the "Save" button.

[0859] 2. Terminal:

[0860] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[0861] 3. Server:

[0862] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[0863] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[0864] Leveraging expertise in a common environment

[0865] 1. User:

[0866] Enter your tax and legal questions in a common environment and click the "Submit" button.

[0867] 2. Terminal:

[0868] The user's question is sent to a common environment and the relevant information is searched for.

[0869] 3. Server:

[0870] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0871] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[0872] Audio and video data management

[0873] 1. User:

[0874] Record and upload business content and know-how in audio or video format.

[0875] 2. Terminal:

[0876] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[0877] 3. Server:

[0878] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0879] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[0880] Manage your billing information

[0881] 1. Server:

[0882] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[0883] 2. Terminal:

[0884] Display billing information to administrators and provide payment links.

[0885] 3. Admin user:

[0886] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[0887] Utilizing the Emotion Engine

[0888] 1. User:

[0889] Interact with the system through voice and text.

[0890] 2. Emotion Engine:

[0891] Analyzes the user's voice and text to recognize emotions.

[0892] 3. Server:

[0893] It tailors its response based on the perceived emotion and provides personalized feedback to the user.

[0894] 4. Terminal:

[0895] The tailored response is displayed to the user, improving the user's experience.

[0896] Example: If a user gives a speech input that indicates frustration to the system, the emotion engine will recognize that emotion and the system will provide a more supportive response.

[0897] Accumulation and use of emotional data

[0898] 1. Emotion Engine:

[0899] Emotional data for each user is accumulated and saved as a history.

[0900] 2. Server:

[0901] The accumulated emotional data is analyzed to extract trends and patterns for individual users.

[0902] 3. Terminal:

[0903] The analysis results are used to further personalize the user experience and provide adaptive feedback.

[0904] Example: If a user regularly expresses negative sentiment towards the same question, the system can take action to improve the answer to that question and increase user satisfaction.

[0905] In this way, the present invention combines data management for each SME, utilization of common expertise, and an emotion engine to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[0906] The processing flow will be explained below.

[0907] Setting up a dedicated environment

[0908] Step 1:

[0909] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[0910] Step 2:

[0911] The server initializes the administrator account and provides the credentials to the company.

[0912] Step 3:

[0913] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[0914] Step 4:

[0915] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[0916] Data entry and management

[0917] Step 1:

[0918] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[0919] Step 2:

[0920] A user uses a particular data entry form and clicks the "Save" button.

[0921] Step 3:

[0922] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[0923] Step 4:

[0924] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[0925] Leveraging expertise in a common environment

[0926] Step 1:

[0927] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[0928] Step 2:

[0929] The terminal transmits the user's question to the common environment and searches for relevant information.

[0930] Step 3:

[0931] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[0932] Audio and video data management

[0933] Step 1:

[0934] Users record and upload their business operations and know-how in audio and video formats.

[0935] Step 2:

[0936] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[0937] Step 3:

[0938] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[0939] Manage your billing information

[0940] Step 1:

[0941] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[0942] Step 2:

[0943] The server generates an invoice and notifies the administrative user.

[0944] Step 3:

[0945] The terminal displays billing information to the administrator and provides a payment link.

[0946] Step 4:

[0947] Admin user reviews invoices and makes payments online.

[0948] Step 5:

[0949] The server updates the records in the system when the payment is completed.

[0950] Utilizing the Emotion Engine

[0951] Step 1:

[0952] The user interacts with the system through voice and text.

[0953] Step 2:

[0954] The emotion engine analyzes the user's voice and text to recognize emotions.

[0955] Step 3:

[0956] The server tailors the response based on the recognized emotion and provides personalized feedback to the user.

[0957] Step 4:

[0958] The terminal displays the adjusted response to the user, improving the user experience.

[0959] Accumulation and use of emotional data

[0960] Step 1:

[0961] The emotion engine accumulates each user's emotion data and saves it as a history.

[0962] Step 2:

[0963] The server analyzes the accumulated emotional data and extracts trends and patterns for individual users.

[0964] Step 3:

[0965] The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[0966] Example 2

[0967] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0968] Small and medium-sized enterprises (SMEs) have a wide range of tasks, including data management, utilizing specialized knowledge, and managing billing information, making it difficult to perform these tasks efficiently. Furthermore, they lack a system that can properly recognize employee emotions and provide personalized feedback. Therefore, there is a need for a system that can improve both work efficiency and employee satisfaction.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0970] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for saving and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for recognizing user emotions, accumulating and analyzing emotion data, and means for adjusting responses and providing personalized feedback based on the recognized emotions, thereby enabling small and medium-sized enterprises to efficiently manage data and utilize knowledge, and to provide personalized feedback according to user emotions.

[0971] "Company-specific database" refers to an individual database set up by each company to securely store and manage its own data.

[0972] "Authentication Information" means a username, password, or other authentication means to grant access to a System.

[0973] "Confidential data" refers to important information held within a company, such as sales data, customer information, and business know-how.

[0974] A "specialized knowledge base" refers to a database that compiles information and guidelines on a specific field.

[0975] "Means for uploading and managing audio and video" refers to the functionality that allows users to upload audio and video data to the system and store and manage it.

[0976] "Billing Information" refers to the fees calculated based on each company's service usage record and information related to the payment thereof.

[0977] "Means for recognizing emotions and storing and analyzing emotional data" refers to the function of analyzing emotions from the user's voice and text, and collecting, saving, and analyzing that data.

[0978] "Means for adjusting responses and providing personalized feedback based on recognized emotions" refers to the ability of the system to dynamically generate responses and provide individualized feedback based on the user's emotions.

[0979] This invention is a system that provides a more personalized user experience and promotes business efficiency by combining a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions. This system includes the following elements.

[0980] Overall system overview

[0981] The system provides each small and medium-sized enterprise with a dedicated database to securely manage confidential data for each company. It also accesses a common dedicated knowledge base and includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[0982] Setting up a dedicated environment

[0983] 1. Server:

[0984] We create a dedicated database for each small and medium-sized business and provide an environment where corporate data can be safely stored, using database systems such as MySQL and PostgreSQL.

[0985] Initialize an administrator account and provide credentials to the company, which are managed using a framework such as Django or Ruby on Rails.

[0986] 2. User (Administrator):

[0987] After logging in for the first time, you can add accounts for other employees and set the appropriate permissions for each account. Administrative functions are performed through the web interface.

[0988] 3. Server:

[0989] Employee account authentication information is managed using security services such as Auth0 and Firebase, and security measures are implemented to prevent unauthorized access.

[0990] Data entry and management

[0991] 1. User:

[0992] Log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a data entry form built on a front-end framework (e.g., React, Vue.js).

[0993] 2. Terminal:

[0994] The entered data is sent to a company-specific RESTful API endpoint, and a save request is sent to the server. The communication protocol is HTTP.

[0995] 3. Server:

[0996] Save the data received from the device in a company-specific database and notify the device when the save is complete. Use a backend framework (e.g., Express.js, Spring Boot).

[0997] As a specific example of how it works, users can input daily sales data and the system will save this data in real time, allowing them to instantly grasp sales trends and helping to improve business operations.

[0998] Leveraging expertise in a common environment

[0999] 1. User:

[1000] Enter your tax and legal questions in a common environment and click the "Submit" button.

[1001] 2. Terminal:

[1002] The user's question is sent to a common environment and the relevant information is searched for.

[1003] 3. Server:

[1004] It retrieves relevant information from a dedicated knowledge base in the common environment and sends it back to the device, using a wiki or other information resource.

[1005] As a specific example of how it works, when a user asks "How to file tax returns the latest?", the system retrieves the latest information from the common environment and provides detailed guidance.

[1006] Audio and video data management

[1007] 1. User:

[1008] Record and upload business content and know-how in audio or video format.

[1009] 2. Terminal:

[1010] The recording and video data is sent to the company's dedicated cloud storage (e.g., Amazon S3) and a storage request is sent to the server.

[1011] 3. Server:

[1012] The uploaded audio data is converted to text using the Google Cloud Speech-to-Text API, and metadata is generated and indexed in Elasticsearch.

[1013] As a specific example of how it works, a user can record a video of the assembly procedure for a new product, and the system will analyze the video, making it easy for other employees to search and refer to.

[1014] Manage your billing information

[1015] 1. Server:

[1016] Calculate the billing amount for each company based on monthly usage. Generate an invoice using Stripe API and notify the administrator user.

[1017] 2. Terminal:

[1018] Display billing information to administrators and provide payment links.

[1019] 3. Admin user:

[1020] View your invoice and pay online. Once payment is complete, your records will be automatically updated in the system.

[1021] Utilizing the Emotion Engine and Response Adjustment

[1022] 1. User:

[1023] Interact with the system via voice and text.

[1024] 2. Emotion Engine:

[1025] It uses IBM Watson's Natural Language Understanding API to analyze the user's voice and text and recognize emotions.

[1026] 3. Server:

[1027] It tailors responses based on perceived emotions and provides personalized feedback using generative AI models.

[1028] 4. Terminal:

[1029] The tailored response is displayed to the user, improving the user's experience.

[1030] As a specific example of how this works, if a user makes a speech input to the system that indicates frustration, the emotion engine will recognize the emotion and the system will generate and display a more supportive and understanding response.

[1031] Accumulation and analysis of emotional data

[1032] 1. Emotion Engine:

[1033] Emotion data for each user is periodically accumulated.

[1034] 2. Server:

[1035] The accumulated emotional data is analyzed using machine learning models (e.g., TensorFlow, PyTorch) to extract trends and patterns for individual users.

[1036] 3. Terminal:

[1037] Based on the analysis results, the user experience can be further personalized and adaptive feedback can be provided.

[1038] As a specific example of how it works, if a user repeatedly has a negative reaction to a particular question, that emotional data is accumulated and analyzed, and the system improves the answer to that question.

[1039] An example prompt might be, "What is the latest information on tax filing? I currently run a small business and need advice on filing my 2023 taxes."

[1040] As described above, the present invention combines data management for each SME with the utilization of common expertise, as well as an emotion engine, to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[1041] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1042] Step 1: User authentication

[1043] Input: The user enters their username and password on the login screen.

[1044] Processing: The device sends the input information to the Auth0 API. Auth0 verifies the authentication information and generates an authentication token.

[1045] Output: The server receives the authentication token and returns it to the device indicating that the user has been authenticated.

[1046] What it does: When a user enters their credentials on the login screen, the system verifies them through Auth0 and redirects them to the dashboard if authentication is successful.

[1047] Step 2: Enter and save data

[1048] Input: Users enter sensitive data, such as sales data or customer information, into data entry forms.

[1049] Processing: The device sends the input data to a company-specific server using a RESTful API, which stores the data in a MySQL database.

[1050] Output: The server notifies the device that the save is complete.

[1051] Specific behavior: When a user enters sales data into a React-based data entry form and clicks the "Save" button, the data is sent to the server and saved in the database.

[1052] Step 3: Find expertise

[1053] Input: User enters tax or legal question in a common environment and clicks "Submit" button.

[1054] Processing: The device sends the query to the common environment search API to search for related information.

[1055] Output: The server retrieves the relevant information from the knowledge base and returns the search results to the terminal.

[1056] What it does: When a user types "latest tax filing method" into the search bar and hits the submit button, the system displays the latest relevant tax guidelines.

[1057] Step 4: Managing audio and video data

[1058] Input: Users record and record their work content and know-how in audio or video format and upload it to the system.

[1059] Processing: The device sends the audio and video data to the company's dedicated cloud storage and sends a storage request to the server. The server analyzes the audio data and converts it into text.

[1060] Output: The server generates the parsed text and metadata and indexes it into Elasticsearch.

[1061] Specific operation: When a user records a video of their business know-how and uploads it to the system, the audio is automatically converted into text and saved in a searchable format.

[1062] Step 5: Manage your billing information

[1063] Input: The server generates billing information based on monthly usage data.

[1064] Processing: The server uses the Stripe API to generate invoices for each company and notify the admin user.

[1065] Output: The terminal displays the billing information to the administrator and provides a payment link.

[1066] Specific operation: At the end of each month, the system will compile usage data, automatically generate an invoice through Stripe, and notify the administrator user. The administrator can then log in, check the invoice, and pay online.

[1067] Step 6: Leverage your emotional engine and adjust your response

[1068] Input: The user interacts with the system through voice and text.

[1069] Processing: The emotion engine uses IBM Watson APIs to analyze the user's emotions. The server tailors responses based on the recognized emotions and provides personalized feedback using generative AI models.

[1070] Output: The device displays the adjusted response to the user, improving the user's experience.

[1071] How it works: When a user inputs speech that expresses dissatisfaction with the system, the emotion engine recognizes the emotion and the system provides a supportive response.

[1072] Step 7: Collect and analyze emotional data

[1073] Input: The emotion engine periodically collects emotion data for each user.

[1074] Processing: The server analyzes the accumulated emotional data using machine learning models to extract trends and patterns of the user's emotions.

[1075] Output: The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[1076] What it does: If a user expresses negative sentiment towards a particular question, that data is stored in the system to improve future answers.

[1077] (Application example 2)

[1078] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1079] The purpose of this invention is to improve business efficiency and productivity in small and medium-sized enterprises. Previous systems did not adequately utilize company-specific data management and specialized knowledge, making it difficult to optimize business processes. Furthermore, there was a lack of personalized responses and support that took employee emotions into consideration, leaving the issue of improving the working environment unresolved.

[1080] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1081] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for interpreting voice instructions and dynamically changing work plans, and means for recognizing user emotions using an emotion engine and adjusting responses, thereby enabling more efficient data management, optimization of business processes, and personalized support based on employee emotions.

[1082] A "company-specific database" is a database in which data is securely stored and managed in a format that each company can use independently.

[1083] "Authentication information" is identification information required when a user accesses a system, and is information used to ensure security.

[1084] "Confidential data" refers to important information held by a company that must not be leaked to the outside.

[1085] A "dedicated knowledge base" is a database in which specialized knowledge and information are accumulated and managed, and can be shared when necessary.

[1086] "Audio and video uploading and management" refers to the process by which users provide audio and video data to the system, which then stores and organizes that data.

[1087] "Billing information" refers to information regarding fees and payments calculated based on usage history.

[1088] "Means for interpreting voice instructions and dynamically changing work plans" refers to a function that understands instructions provided by voice and automatically adjusts the schedule and content of work.

[1089] An "emotion engine" is a system that analyzes the user's emotions and generates and adjusts appropriate responses based on those emotions.

[1090] The present invention aims to improve business efficiency and productivity in small and medium-sized enterprises. Specific embodiments will be described below.

[1091] Overall system configuration

[1092] The system consists of the following main elements:

[1093] Server Roles

[1094] The server creates a dedicated database for each company, manages authentication information for each company, stores and manages confidential data, accesses a common dedicated knowledge base, uploads and manages audio and video data, and generates and manages billing information for each company.

[1095] Device Role

[1096] The device provides an interface for users to issue voice commands, interprets the voice commands, dynamically changes the task plan, and uses an emotion engine to recognize the user's emotions and adjust responses accordingly.

[1097] User Roles

[1098] Users interact with the system by giving instructions to it through voice or text, for example, they can ask the system to explain a specific work procedure.

[1099] Main processing flow

[1100] Speech recognition processing

[1101] The device uses voice recognition technology to analyze user instructions, using a microphone and a Raspberry Pi to convert voice data into text using the SpeechRecognition library.

[1102] Emotion recognition processing

[1103] The emotion engine analyzes emotions from the user's voice or text using an external emotion recognition API, for example, to determine whether the user is frustrated based on the content of their voice instructions.

[1104] Data storage and sharing

[1105] The audio and video data collected from users is stored on a cloud server (e.g., AWS, Google Cloud). PostgreSQL or MongoDB is used as the database. The data is managed in a way that makes it properly categorized and searchable.

[1106] Exception handling assistance

[1107] If the emotion engine detects negative emotions from the user, the system will provide guidance for immediate action and automatically initiate assistance through technical staff if support is needed.

[1108] Specific examples

[1109] For example, in a scenario where a worker is explaining the production procedure for a new part in a factory, the worker might give the robot a voice command such as "Explain the production procedure for the new part." In this case:

[1110] 1. The device analyzes the voice commands and converts them into text.

[1111] 2. The emotion engine recognizes the user's emotions and, if it determines that the worker is feeling frustrated, it presents detailed instructions in video format.

[1112] 3. Detailed visual guides are provided to help workers understand better.

[1113] Prompt Sentence Examples

[1114] Here are some example prompts to input to a generative AI model:

[1115] text

[1116] A worker commands the robot to "explain the production procedure for a new part." The emotion engine recognizes that the worker is expressing frustration. In this case, the robot displays a detailed video of the procedure to help the worker understand.

[1117] In this way, the system provided by the present invention not only achieves data management and business process optimization for small and medium-sized enterprises, but also provides flexible support according to the emotional state of employees.

[1118] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1119] Step 1:

[1120] The user gives a voice command such as "Explain the production procedure for the new part."

[1121] Input: A spoken command from the user.

[1122] Output: Audio data.

[1123] Specific action: The user speaks into the device's microphone.

[1124] Step 2:

[1125] The terminal uses voice recognition software to convert the voice data into text.

[1126] Input: Audio data.

[1127] Output: Textual instructions.

[1128] Specific operation: Uses the SpeechRecognition library to convert voice data into text. Uses devices such as Raspberry Pi.

[1129] Step 3:

[1130] The device sends the converted text to the emotion engine's API to analyze the user's emotions.

[1131] Input: Instructions in text format.

[1132] Output: Sentiment analysis results.

[1133] Specific operation: The device sends text data to the emotion recognition API, and the emotion engine analyzes the emotion.

[1134] Step 4:

[1135] If the emotion engine detects frustration, the device will display supportive instructions.

[1136] Input: Sentiment analysis results.

[1137] Output: Instructions and procedures for support.

[1138] Specific operation: Based on the analysis results, the device displays an easy-to-understand visual guide to the user.

[1139] Step 5:

[1140] The server stores the emotion engine's analysis results and the contents of the voice instructions as a log.

[1141] Input: Sentiment analysis results and instructions in text format.

[1142] Output: Log data.

[1143] Specific operation: Works with a cloud server (e.g. AWS, Google Cloud) to store results in a database.

[1144] Step 6:

[1145] The user follows the provided visual guide as they work.

[1146] Enter: a visual guide.

[1147] Output: User's work progress.

[1148] Specific operation: The user learns and executes the production procedure for a new part by following the video and text guide displayed on the device screen.

[1149] By providing a detailed explanation of the processing performed at each step and clearly showing the overall flow of the system, the quality of the user experience can be improved and business efficiency for small and medium-sized enterprises can be improved.

[1150] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1151] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1152] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1153] [Third embodiment]

[1154] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1155] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1156] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1157] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1158] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1159] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1160] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1161] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1162] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1163] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1164] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1165] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1166] This invention is a generative AI platform for small and medium-sized enterprises, which supports the digitalization and productivity improvement of small and medium-sized enterprises by providing dedicated environments for each company and a common environment.

[1167] Overall system overview

[1168] The system provides each small and medium-sized enterprise with a dedicated database, safely manages confidential data for each company, and creates an environment where common expertise can be utilized. It also includes a variety of functions, such as managing audio and video data, sharing business know-how, and processing billing information.

[1169] Setting up a dedicated environment

[1170] 1. Server:

[1171] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[1172] Initialize the administrator account and provide credentials to the company.

[1173] 2. User (Administrator):

[1174] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[1175] 3. Server:

[1176] Manage employee account credentials and implement security measures to prevent unauthorized access.

[1177] Data entry and management

[1178] 1. User:

[1179] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[1180] Use a specific data entry form and click the "Save" button.

[1181] 2. Terminal:

[1182] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[1183] 3. Server:

[1184] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[1185] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[1186] Leveraging expertise in a common environment

[1187] 1. User:

[1188] Enter your tax and legal questions in a common environment and click the "Submit" button.

[1189] 2. Terminal:

[1190] The user's question is sent to a common environment and the relevant information is searched for.

[1191] 3. Server:

[1192] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[1193] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[1194] Audio and video data management

[1195] 1. User:

[1196] Record and upload business content and know-how in audio or video format.

[1197] 2. Terminal:

[1198] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[1199] 3. Server:

[1200] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[1201] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[1202] Manage your billing information

[1203] 1. Server:

[1204] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[1205] 2. Terminal:

[1206] Display billing information to administrators and provide payment links.

[1207] 3. Admin user:

[1208] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[1209] Example: At the end of each month, the server aggregates the usage history and generates the appropriate invoice, which the administrator can then view and easily complete the payment.

[1210] In this way, the present invention combines data management for each small and medium-sized enterprise with the utilization of common expertise to achieve efficient business operations and accurate billing processing using voice and video, thereby resolving many of the challenges faced by small and medium-sized enterprises and improving productivity.

[1211] The processing flow will be explained below.

[1212] Setting up a dedicated environment

[1213] Step 1:

[1214] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[1215] Step 2:

[1216] The server initializes the administrator account and provides the credentials to the company.

[1217] Step 3:

[1218] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[1219] Step 4:

[1220] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[1221] Data entry and management

[1222] Step 1:

[1223] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[1224] Step 2:

[1225] A user uses a particular data entry form and clicks the "Save" button.

[1226] Step 3:

[1227] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[1228] Step 4:

[1229] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[1230] Leveraging expertise in a common environment

[1231] Step 1:

[1232] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[1233] Step 2:

[1234] The terminal transmits the user's question to the common environment and searches for relevant information.

[1235] Step 3:

[1236] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[1237] Audio and video data management

[1238] Step 1:

[1239] Users record and upload their business operations and know-how in audio and video formats.

[1240] Step 2:

[1241] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[1242] Step 3:

[1243] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[1244] Manage your billing information

[1245] Step 1:

[1246] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[1247] Step 2:

[1248] The server generates an invoice and notifies the administrative user.

[1249] Step 3:

[1250] The terminal displays billing information to the administrator and provides a payment link.

[1251] Step 4:

[1252] Admin user reviews invoices and makes payments online.

[1253] Step 5:

[1254] The server updates the records in the system when the payment is completed.

[1255] Example 1

[1256] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1257] Currently, many small and medium-sized enterprises have not made much progress in digitalization, and are facing challenges in business efficiency and productivity. Furthermore, it is difficult to utilize the different data management and expertise that each company has, and in particular, the management of audio and video data and proper billing processing are insufficient. As a result, they are unable to obtain real-time information or appropriate advice for daily operations and management decisions, which is hindering productivity improvement.

[1258] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1259] In this invention, the server includes a means for generating a database for storing data specific to each company, a means for managing authentication information for each company, a means for inputting, saving, and managing confidential data for each company, a means for accessing a common knowledge base and providing related information, a means for uploading, managing, and analyzing audio and video data, and a means for generating and managing billing information based on each company's usage history. This allows small and medium-sized enterprises to promote digitalization, obtain information in real time, and receive appropriate advice. It also improves business efficiency and accurate billing processing, significantly increasing productivity.

[1260] A "database" is a system for managing, storing, and efficiently manipulating a collection of data created specifically for each company.

[1261] "Authentication information" refers to information required for a user to access a system, and is primarily composed of a username, password, security token, and the like.

[1262] "Confidential data" refers to data that should only be used within a specific company, such as a company's sales data or customer information, and must be managed with high security.

[1263] A "knowledge base" is a database in which various specialized knowledge and information are organized and stored, and is used to provide appropriate answers to user questions.

[1264] "Audio data" is a recording of a user's work content or know-how, and is mainly saved in the form of an audio file.

[1265] "Video data" refers to work content or know-how recorded by a user, and is mainly saved in the form of a video file.

[1266] "Uploading" is the process by which a user sends data from their device to a server and stores it.

[1267] "Analysis" is the process of analyzing uploaded audio or video, converting the content into text, and generating metadata.

[1268] "Metadata" is auxiliary information added to facilitate data management and search, and is used for classification and search.

[1269] "Billing information" is data including the billing amount calculated based on the usage record of each company and its details, and is managed as a bill.

[1270] This invention relates to a generative AI platform for small and medium-sized enterprises, which supports their digitalization and productivity improvement by providing dedicated and shared environments for each company. This platform integrates a dedicated database, secure authentication management, confidential data processing, utilization of specialized knowledge, audio and video data management, and proper processing of billing information.

[1271] Setting up a dedicated environment

[1272] The server creates a dedicated database for each small or medium-sized business, allowing data to be kept separate and preventing it from being mixed with other companies' data. The database contains tables for sales, customers, invoices, etc., and can be managed exclusively by each company.

[1273] The server then initializes an administrator account and sends the credentials to the company, using the bcrypt library for password hashing. After receiving the administrator account, the company administrator can add employee accounts and set the necessary permissions after the first login.

[1274] Additionally, the server uses an ID management service such as Auth0 to securely manage authentication information and performs authentication using JWT tokens, implementing advanced security measures to prevent unauthorized access.

[1275] Data entry and management

[1276] Users log in to the company's dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form, and by clicking the "Save" button, the data is sent to the server.

[1277] The device sends the entered data to a company-specific API endpoint and makes a save request to the server. This data is sent using HTTPS, ensuring security. The server receives the data, validates it, and saves it in a MySQL database. The user is notified when the save is successful.

[1278] Leveraging expertise in a common environment

[1279] Users access the common environment and enter their tax or legal questions into a text form, enter a specific question such as "How do I file my taxes the latest?", and click the "Submit" button.

[1280] The device sends the question to the common environment and generates a prompt to search for the relevant information in the knowledge base. The server retrieves the appropriate information from the knowledge base and returns it to the user, allowing the user to instantly access the latest legal and tax information.

[1281] Audio and video data management

[1282] Users can record their work details and know-how in audio and video formats. For example, they can record a video of the assembly procedure for a new product on their smartphone and upload it to the system.

[1283] The device sends the uploaded audio and video data to a company-specific storage (e.g., AWS S3), where the server converts the audio to text using AWS Transcribe and stores it as metadata in DynamoDB, making it easily searchable and browsable later.

[1284] Manage your billing information

[1285] The server aggregates the usage data for each company monthly, calculates the billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator user.

[1286] The terminal then displays the billing information to the administrator and provides an online payment link, which uses the Stripe payment gateway and automatically updates records in the system once payment is completed. Admin users can view their invoices and quickly pay online.

[1287] Prompt Sentence Examples

[1288] 1. "How do I enter new sales data?"

[1289] 2. "How do I file my taxes the latest?"

[1290] By implementing these procedures, the present invention solves many of the problems faced by small and medium-sized enterprises, and improves business efficiency and productivity.

[1291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1292] Setting up a dedicated environment

[1293] Step 1:

[1294] The server creates a database for each small or medium-sized business. Specifically, it uses a MySQL database with a different schema for each company. For example, for Company A, it creates tables for sales, customers, invoices, etc.

[1295] Input: Small and Medium Enterprise Information

[1296] Output: Company-specific database

[1297] Step 2:

[1298] The server initializes an administrator account, hashes the credentials using the bcrypt library, and then emails the resulting credentials to the company.

[1299] Input: Administrator information

[1300] Output: Admin account with hashed credentials

[1301] Step 3:

[1302] The user (administrator) logs into the dedicated environment and adds accounts for other employees. The administrator sets the appropriate permissions for each account and grants the access rights required for work.

[1303] Input: Employee information

[1304] Output: Permissioned employee accounts

[1305] Step 4:

[1306] The server manages employee account authentication information, uses identity management services such as Auth0 to prevent unauthorized access, generates JWT tokens, and implements security measures.

[1307] Input: Employee Credentials

[1308] Output: Secure credentials

[1309] Data entry and management

[1310] Step 1:

[1311] Users log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form and then the user clicks the "Save" button.

[1312] Input: Sales data, customer information

[1313] Output: Form data

[1314] Step 2:

[1315] The device sends the data entered by the user in JSON format to the enterprise API endpoint and makes a save request to the server. This request is sent using HTTPS, ensuring security.

[1316] Input: Form Data

[1317] Output: API request

[1318] Step 3:

[1319] The server validates the data received from the terminal and saves it in the appropriate table (sales, customer, etc.) in the dedicated database. It notifies the terminal that saving is complete.

[1320] Input: API request data

[1321] Output: Database entry, save success notification

[1322] Leveraging expertise in a common environment

[1323] Step 1:

[1324] Users access the common environment, enter a question, such as "How do I file my taxes the latest?" into a text form, and click the "Submit" button.

[1325] Input: Question

[1326] Output: Question submitted

[1327] Step 2:

[1328] The terminal transmits the transmitted question to the common environment and generates a prompt to search for the relevant information within the knowledge base.

[1329] Input: Submitted question

[1330] Output: Search prompt

[1331] Step 3:

[1332] The server retrieves relevant information from the knowledge base, generates an answer, and sends it back to the device, providing detailed guidance based on tax or legal information, for example.

[1333] Input: Search prompt

[1334] Output: Answer information

[1335] Audio and video data management

[1336] Step 1:

[1337] Users record and record their work content and know-how in audio or video format and upload the files to the system.

[1338] Input: Audio files, video files

[1339] Output: Upload request

[1340] Step 2:

[1341] The device sends the uploaded audio and video data to the company's dedicated storage and makes a storage request to the server, for example, AWS S3, to store the data securely.

[1342] Input: Upload request

[1343] Output: Save request to storage

[1344] Step 3:

[1345] The server converts the audio to text using AWS Transcribe and stores the text data along with metadata in DynamoDB, which makes it easier to search and categorize later.

[1346] Input: Storage data

[1347] Output: Text data, metadata

[1348] Manage your billing information

[1349] Step 1:

[1350] The server aggregates each company's monthly usage data, calculates the appropriate billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator.

[1351] Input: Monthly usage record

[1352] Output: Invoice PDF

[1353] Step 2:

[1354] The terminal displays billing information to the administrator and provides an online payment link, using the Stripe payment gateway.

[1355] Input: Invoice PDF, administrator information

[1356] Output: Payment link

[1357] Step 3:

[1358] The admin user reviews the invoice and clicks on the provided link to complete the payment online. Once the payment is successful, the transaction record is automatically updated in the system.

[1359] Input: Payment information

[1360] Output: Updated transaction record

[1361] (Application example 1)

[1362] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1363] In recent years, the slow pace of digitalization among small and medium-sized enterprises has become a problem. While there is a demand for improved productivity, technical resources are often lacking. In particular, for companies that use automated vehicles, managing and optimizing vehicle operation data requires a great deal of time and effort, which leads to a decline in business efficiency. In addition, it is difficult to safely manage confidential data and share specialized knowledge, and audio and video data are not being effectively utilized. There is a need for a system that can solve these issues and support the digitalization and productivity improvement of small and medium-sized enterprises.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1365] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for collecting operation data of automatically controlled vehicles and transmitting it to a company-dedicated server, and means for analyzing the operation data and optimizing operation plans in real time. This enables efficient management and optimization of operation data of automatically controlled vehicles, enabling digitalization and improved productivity for small and medium-sized enterprises.

[1366] A "database" is a system for systematically organizing, storing, and managing information.

[1367] "Authentication information" is information used to verify the authenticity of a user or system.

[1368] "Confidential data" refers to important data that could cause harm if leaked to the outside.

[1369] A "knowledge base" is a database that systematically compiles specific knowledge and information.

[1370] "Audio data" means data that is a digital representation of a recording of spoken words or other sounds.

[1371] "Moving image data" is digital data that expresses movement by playing back a series of still images at regular time intervals.

[1372] "Billing information" is information indicating the cost for providing a service or product.

[1373] An "automated vehicle" is a vehicle that can operate automatically without human operation.

[1374] "Operation data" refers to various data related to operation, such as the vehicle's position, speed, and condition.

[1375] A "server" is a computer system that provides services over a network.

[1376] "Real-time" refers to a state in which information and processing are carried out immediately without delay.

[1377] This invention is a system that collects and manages operational data from autonomous vehicles to support the digitalization and productivity improvement of small and medium-sized enterprises. Specifically, it creates a dedicated database for each company, securely manages authentication information and confidential data, and provides an environment where a common knowledge base can be easily accessed. The system also includes functions such as uploading and managing audio and video data, generating and managing billing information for each company, and collecting, analyzing, and optimizing operational data from autonomous vehicles.

[1378] System Configuration

[1379] 1. Creation and management of a dedicated database for each company

[1380] The server creates a dedicated database for each company and stores confidential data in that database. Each company's authentication information is managed in this database, and security measures are implemented to prevent unauthorized access. For example, an administrator completes the initial login and adds employee accounts, setting appropriate permissions.

[1381] 2. Access to a common, dedicated knowledge base

[1382] Users can use a common, dedicated knowledge base to ask tax and legal questions and get answers in real time, improving work efficiency. For example, by entering a question like "What is the latest tax filing method?" into the system, the latest information is retrieved from the common environment.

[1383] 3. Audio and video data management

[1384] Users record their work and know-how in audio or video format and upload it to the system. The server analyzes the audio data, converts it into text, and generates metadata, which makes it easy for other employees to search and refer to. For example, a video of the assembly procedure for a new product can be recorded and made available to each employee.

[1385] 4. Management of operational data for autonomous vehicles

[1386] Operational data from autonomous vehicles is sent in real time from the vehicle's terminal to a company-dedicated server. This data includes location information, battery level, and operating status. The server analyzes this data and optimizes operation plans in real time. For example, if the battery level is low, it will guide the vehicle to the nearest charging station.

[1387] 5. Generative AI Model and Prompts

[1388] Generative AI models are used to perform various data analyses and optimizations. For example, a prompt such as "Please tell me how to develop a system that manages autonomous vehicle operation data in real time and provides optimal operation plans" can be generated and input into the AI ​​model to achieve advanced analysis.

[1389] Hardware and Software Examples

[1390] Hardware: Servers, smartphones, autonomous vehicle terminals

[1391] Software: Python, requests library, enterprise-specific API endpoints

[1392] The above configuration and means will enable digitalization and productivity improvement in small and medium-sized enterprises.

[1393] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1394] Step 1:

[1395] The user logs into the dedicated environment and inputs vehicle operation data. Specifically, the user enters data such as the vehicle's location, battery level, and operating status into an input form and clicks the "Submit" button. The input data also includes the vehicle's identification information.

[1396] Step 2:

[1397] The device sends the operational data entered by the user to a company-specific API endpoint, where it is packaged in JSON format and sent as an HTTP POST request to the API endpoint. The device processes the data and sends it to the server via a communication protocol.

[1398] Step 3:

[1399] The server analyzes the operational data received from the terminal and stores it in the company's dedicated database. The data is converted into the appropriate format and inserted into the specified table in the database. Once the data has been saved, the server notifies the terminal that the data has been saved successfully.

[1400] Step 4:

[1401] The server performs real-time analysis based on the stored operational data to optimize the operation plan. This analysis involves statistically processing multidimensional data such as location information, battery level, and operation status to select the optimal operation route and charging station. Generative AI models are utilized here.

[1402] Step 5:

[1403] The server sends the analysis results back to the device via a dedicated API endpoint. Specifically, it includes an optimized operation plan and instructions for when emergency charging is required. The device receives this analysis data and notifies the user.

[1404] Step 6:

[1405] The user checks the analysis results notified from the device and modifies the operation plan as necessary. For example, if an instruction to go to a charging station is issued, the user will act promptly in accordance with the instruction. The input in this step is the analysis results, and the output is the user's action.

[1406] The above processing steps seamlessly realize a series of steps including data collection, analysis, optimization, and notification, enabling small and medium-sized enterprises to digitalize and improve productivity.

[1407] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1408] This invention combines a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions to provide a more personalized user experience and promote business efficiency.

[1409] Overall system overview

[1410] The system provides each small and medium-sized enterprise with a dedicated database, safely managing each company's confidential data and creating an environment where they can utilize shared expertise. It also includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[1411] Setting up a dedicated environment

[1412] 1. Server:

[1413] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[1414] Initialize the administrator account and provide credentials to the company.

[1415] 2. User (Administrator):

[1416] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[1417] 3. Server:

[1418] Manage employee account credentials and implement security measures to prevent unauthorized access.

[1419] Data entry and management

[1420] 1. User:

[1421] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[1422] Use a specific data entry form and click the "Save" button.

[1423] 2. Terminal:

[1424] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[1425] 3. Server:

[1426] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[1427] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[1428] Leveraging expertise in a common environment

[1429] 1. User:

[1430] Enter your tax and legal questions in a common environment and click the "Submit" button.

[1431] 2. Terminal:

[1432] The user's question is sent to a common environment and the relevant information is searched for.

[1433] 3. Server:

[1434] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[1435] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[1436] Audio and video data management

[1437] 1. User:

[1438] Record and upload business content and know-how in audio or video format.

[1439] 2. Terminal:

[1440] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[1441] 3. Server:

[1442] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[1443] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[1444] Manage your billing information

[1445] 1. Server:

[1446] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[1447] 2. Terminal:

[1448] Display billing information to administrators and provide payment links.

[1449] 3. Admin user:

[1450] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[1451] Utilizing the Emotion Engine

[1452] 1. User:

[1453] Interact with the system through voice and text.

[1454] 2. Emotion Engine:

[1455] Analyzes the user's voice and text to recognize emotions.

[1456] 3. Server:

[1457] It tailors its response based on the perceived emotion and provides personalized feedback to the user.

[1458] 4. Terminal:

[1459] The tailored response is displayed to the user, improving the user's experience.

[1460] Example: If a user gives a speech input that indicates frustration to the system, the emotion engine will recognize that emotion and the system will provide a more supportive response.

[1461] Accumulation and use of emotional data

[1462] 1. Emotion Engine:

[1463] Emotional data for each user is accumulated and saved as a history.

[1464] 2. Server:

[1465] The accumulated emotional data is analyzed to extract trends and patterns for individual users.

[1466] 3. Terminal:

[1467] The analysis results are used to further personalize the user experience and provide adaptive feedback.

[1468] Example: If a user regularly expresses negative sentiment towards the same question, the system can take action to improve the answer to that question and increase user satisfaction.

[1469] In this way, the present invention combines data management for each SME, utilization of common expertise, and an emotion engine to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[1470] The processing flow will be explained below.

[1471] Setting up a dedicated environment

[1472] Step 1:

[1473] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[1474] Step 2:

[1475] The server initializes the administrator account and provides the credentials to the company.

[1476] Step 3:

[1477] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[1478] Step 4:

[1479] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[1480] Data entry and management

[1481] Step 1:

[1482] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[1483] Step 2:

[1484] A user uses a particular data entry form and clicks the "Save" button.

[1485] Step 3:

[1486] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[1487] Step 4:

[1488] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[1489] Leveraging expertise in a common environment

[1490] Step 1:

[1491] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[1492] Step 2:

[1493] The terminal transmits the user's question to the common environment and searches for relevant information.

[1494] Step 3:

[1495] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[1496] Audio and video data management

[1497] Step 1:

[1498] Users record and upload their business operations and know-how in audio and video formats.

[1499] Step 2:

[1500] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[1501] Step 3:

[1502] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[1503] Manage your billing information

[1504] Step 1:

[1505] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[1506] Step 2:

[1507] The server generates an invoice and notifies the administrative user.

[1508] Step 3:

[1509] The terminal displays billing information to the administrator and provides a payment link.

[1510] Step 4:

[1511] Admin user reviews invoices and makes payments online.

[1512] Step 5:

[1513] The server updates the records in the system when the payment is completed.

[1514] Utilizing the Emotion Engine

[1515] Step 1:

[1516] The user interacts with the system through voice and text.

[1517] Step 2:

[1518] The emotion engine analyzes the user's voice and text to recognize emotions.

[1519] Step 3:

[1520] The server tailors the response based on the recognized emotion and provides personalized feedback to the user.

[1521] Step 4:

[1522] The terminal displays the adjusted response to the user, improving the user experience.

[1523] Accumulation and use of emotional data

[1524] Step 1:

[1525] The emotion engine accumulates each user's emotion data and saves it as a history.

[1526] Step 2:

[1527] The server analyzes the accumulated emotional data and extracts trends and patterns for individual users.

[1528] Step 3:

[1529] The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[1530] Example 2

[1531] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1532] Small and medium-sized enterprises (SMEs) have a wide range of tasks, including data management, utilizing specialized knowledge, and managing billing information, making it difficult to perform these tasks efficiently. Furthermore, they lack a system that can properly recognize employee emotions and provide personalized feedback. Therefore, there is a need for a system that can improve both work efficiency and employee satisfaction.

[1533] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1534] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for saving and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for recognizing user emotions, accumulating and analyzing emotion data, and means for adjusting responses and providing personalized feedback based on the recognized emotions, thereby enabling small and medium-sized enterprises to efficiently manage data and utilize knowledge, and to provide personalized feedback according to user emotions.

[1535] "Company-specific database" refers to an individual database set up by each company to securely store and manage its own data.

[1536] "Authentication Information" means a username, password, or other authentication means to grant access to a System.

[1537] "Confidential data" refers to important information held within a company, such as sales data, customer information, and business know-how.

[1538] A "specialized knowledge base" refers to a database that compiles information and guidelines on a specific field.

[1539] "Means for uploading and managing audio and video" refers to the functionality that allows users to upload audio and video data to the system and store and manage it.

[1540] "Billing Information" refers to the fees calculated based on each company's service usage record and information related to the payment thereof.

[1541] "Means for recognizing emotions and storing and analyzing emotional data" refers to the function of analyzing emotions from the user's voice and text, and collecting, saving, and analyzing that data.

[1542] "Means for adjusting responses and providing personalized feedback based on recognized emotions" refers to the ability of the system to dynamically generate responses and provide individualized feedback based on the user's emotions.

[1543] This invention is a system that provides a more personalized user experience and promotes business efficiency by combining a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions. This system includes the following elements.

[1544] Overall system overview

[1545] The system provides each small and medium-sized enterprise with a dedicated database to securely manage confidential data for each company. It also accesses a common dedicated knowledge base and includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[1546] Setting up a dedicated environment

[1547] 1. Server:

[1548] We create a dedicated database for each small and medium-sized business and provide an environment where corporate data can be safely stored, using database systems such as MySQL and PostgreSQL.

[1549] Initialize an administrator account and provide credentials to the company, which are managed using a framework such as Django or Ruby on Rails.

[1550] 2. User (Administrator):

[1551] After logging in for the first time, you can add accounts for other employees and set the appropriate permissions for each account. Administrative functions are performed through the web interface.

[1552] 3. Server:

[1553] Employee account authentication information is managed using security services such as Auth0 and Firebase, and security measures are implemented to prevent unauthorized access.

[1554] Data entry and management

[1555] 1. User:

[1556] Log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a data entry form built on a front-end framework (e.g., React, Vue.js).

[1557] 2. Terminal:

[1558] The entered data is sent to a company-specific RESTful API endpoint, and a save request is sent to the server. The communication protocol is HTTP.

[1559] 3. Server:

[1560] Save the data received from the device in a company-specific database and notify the device when the save is complete. Use a backend framework (e.g., Express.js, Spring Boot).

[1561] As a specific example of how it works, users can input daily sales data and the system will save this data in real time, allowing them to instantly grasp sales trends and helping to improve business operations.

[1562] Leveraging expertise in a common environment

[1563] 1. User:

[1564] Enter your tax and legal questions in a common environment and click the "Submit" button.

[1565] 2. Terminal:

[1566] The user's question is sent to a common environment and the relevant information is searched for.

[1567] 3. Server:

[1568] It retrieves relevant information from a dedicated knowledge base in the common environment and sends it back to the device, using a wiki or other information resource.

[1569] As a specific example of how it works, when a user asks "How to file tax returns the latest?", the system retrieves the latest information from the common environment and provides detailed guidance.

[1570] Audio and video data management

[1571] 1. User:

[1572] Record and upload business content and know-how in audio or video format.

[1573] 2. Terminal:

[1574] The recording and video data is sent to the company's dedicated cloud storage (e.g., Amazon S3) and a storage request is sent to the server.

[1575] 3. Server:

[1576] The uploaded audio data is converted to text using the Google Cloud Speech-to-Text API, and metadata is generated and indexed in Elasticsearch.

[1577] As a specific example of how it works, a user can record a video of the assembly procedure for a new product, and the system will analyze the video, making it easy for other employees to search and refer to.

[1578] Manage your billing information

[1579] 1. Server:

[1580] Calculate the billing amount for each company based on monthly usage. Generate an invoice using Stripe API and notify the administrator user.

[1581] 2. Terminal:

[1582] Display billing information to administrators and provide payment links.

[1583] 3. Admin user:

[1584] View your invoice and pay online. Once payment is complete, your records will be automatically updated in the system.

[1585] Utilizing the Emotion Engine and Response Adjustment

[1586] 1. User:

[1587] Interact with the system via voice and text.

[1588] 2. Emotion Engine:

[1589] It uses IBM Watson's Natural Language Understanding API to analyze the user's voice and text and recognize emotions.

[1590] 3. Server:

[1591] It tailors responses based on perceived emotions and provides personalized feedback using generative AI models.

[1592] 4. Terminal:

[1593] The tailored response is displayed to the user, improving the user's experience.

[1594] As a specific example of how this works, if a user makes a speech input to the system that indicates frustration, the emotion engine will recognize the emotion and the system will generate and display a more supportive and understanding response.

[1595] Accumulation and analysis of emotional data

[1596] 1. Emotion Engine:

[1597] Emotion data for each user is periodically accumulated.

[1598] 2. Server:

[1599] The accumulated emotional data is analyzed using machine learning models (e.g., TensorFlow, PyTorch) to extract trends and patterns for individual users.

[1600] 3. Terminal:

[1601] Based on the analysis results, the user experience can be further personalized and adaptive feedback can be provided.

[1602] As a specific example of how it works, if a user repeatedly has a negative reaction to a particular question, that emotional data is accumulated and analyzed, and the system improves the answer to that question.

[1603] An example prompt might be, "What is the latest information on tax filing? I currently run a small business and need advice on filing my 2023 taxes."

[1604] As described above, the present invention combines data management for each SME with the utilization of common expertise, as well as an emotion engine, to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[1605] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1606] Step 1: User authentication

[1607] Input: The user enters their username and password on the login screen.

[1608] Processing: The device sends the input information to the Auth0 API. Auth0 verifies the authentication information and generates an authentication token.

[1609] Output: The server receives the authentication token and returns it to the device indicating that the user has been authenticated.

[1610] What it does: When a user enters their credentials on the login screen, the system verifies them through Auth0 and redirects them to the dashboard if authentication is successful.

[1611] Step 2: Enter and save data

[1612] Input: Users enter sensitive data, such as sales data or customer information, into data entry forms.

[1613] Processing: The device sends the input data to a company-specific server using a RESTful API, which stores the data in a MySQL database.

[1614] Output: The server notifies the device that the save is complete.

[1615] Specific behavior: When a user enters sales data into a React-based data entry form and clicks the "Save" button, the data is sent to the server and saved in the database.

[1616] Step 3: Find expertise

[1617] Input: User enters tax or legal question in a common environment and clicks "Submit" button.

[1618] Processing: The device sends the query to the common environment search API to search for related information.

[1619] Output: The server retrieves the relevant information from the knowledge base and returns the search results to the terminal.

[1620] What it does: When a user types "latest tax filing method" into the search bar and hits the submit button, the system displays the latest relevant tax guidelines.

[1621] Step 4: Managing audio and video data

[1622] Input: Users record and record their work content and know-how in audio or video format and upload it to the system.

[1623] Processing: The device sends the audio and video data to the company's dedicated cloud storage and sends a storage request to the server. The server analyzes the audio data and converts it into text.

[1624] Output: The server generates the parsed text and metadata and indexes it into Elasticsearch.

[1625] Specific operation: When a user records a video of their business know-how and uploads it to the system, the audio is automatically converted into text and saved in a searchable format.

[1626] Step 5: Manage your billing information

[1627] Input: The server generates billing information based on monthly usage data.

[1628] Processing: The server uses the Stripe API to generate invoices for each company and notify the admin user.

[1629] Output: The terminal displays the billing information to the administrator and provides a payment link.

[1630] Specific operation: At the end of each month, the system will compile usage data, automatically generate an invoice through Stripe, and notify the administrator user. The administrator can then log in, check the invoice, and pay online.

[1631] Step 6: Leverage your emotional engine and adjust your response

[1632] Input: The user interacts with the system through voice and text.

[1633] Processing: The emotion engine uses IBM Watson APIs to analyze the user's emotions. The server tailors responses based on the recognized emotions and provides personalized feedback using generative AI models.

[1634] Output: The device displays the adjusted response to the user, improving the user's experience.

[1635] How it works: When a user inputs speech that expresses dissatisfaction with the system, the emotion engine recognizes the emotion and the system provides a supportive response.

[1636] Step 7: Collect and analyze emotional data

[1637] Input: The emotion engine periodically collects emotion data for each user.

[1638] Processing: The server analyzes the accumulated emotional data using machine learning models to extract trends and patterns of the user's emotions.

[1639] Output: The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[1640] What it does: If a user expresses negative sentiment towards a particular question, that data is stored in the system to improve future answers.

[1641] (Application example 2)

[1642] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1643] The purpose of this invention is to improve business efficiency and productivity in small and medium-sized enterprises. Previous systems did not adequately utilize company-specific data management and specialized knowledge, making it difficult to optimize business processes. Furthermore, there was a lack of personalized responses and support that took employee emotions into consideration, leaving the issue of improving the working environment unresolved.

[1644] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1645] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for interpreting voice instructions and dynamically changing work plans, and means for recognizing user emotions using an emotion engine and adjusting responses, thereby enabling more efficient data management, optimization of business processes, and personalized support based on employee emotions.

[1646] A "company-specific database" is a database in which data is securely stored and managed in a format that each company can use independently.

[1647] "Authentication information" is identification information required when a user accesses a system, and is information used to ensure security.

[1648] "Confidential data" refers to important information held by a company that must not be leaked to the outside.

[1649] A "dedicated knowledge base" is a database in which specialized knowledge and information are accumulated and managed, and can be shared when necessary.

[1650] "Audio and video uploading and management" refers to the process by which users provide audio and video data to the system, which then stores and organizes that data.

[1651] "Billing information" refers to information regarding fees and payments calculated based on usage history.

[1652] "Means for interpreting voice instructions and dynamically changing work plans" refers to a function that understands instructions provided by voice and automatically adjusts the schedule and content of work.

[1653] An "emotion engine" is a system that analyzes the user's emotions and generates and adjusts appropriate responses based on those emotions.

[1654] The present invention aims to improve business efficiency and productivity in small and medium-sized enterprises. Specific embodiments will be described below.

[1655] Overall system configuration

[1656] The system consists of the following main elements:

[1657] Server Roles

[1658] The server creates a dedicated database for each company, manages authentication information for each company, stores and manages confidential data, accesses a common dedicated knowledge base, uploads and manages audio and video data, and generates and manages billing information for each company.

[1659] Device Role

[1660] The device provides an interface for users to issue voice commands, interprets the voice commands, dynamically changes the task plan, and uses an emotion engine to recognize the user's emotions and adjust responses accordingly.

[1661] User Roles

[1662] Users interact with the system by giving instructions to it through voice or text, for example, they can ask the system to explain a specific work procedure.

[1663] Main processing flow

[1664] Speech recognition processing

[1665] The device uses voice recognition technology to analyze user instructions, using a microphone and a Raspberry Pi to convert voice data into text using the SpeechRecognition library.

[1666] Emotion recognition processing

[1667] The emotion engine analyzes emotions from the user's voice or text using an external emotion recognition API, for example, to determine whether the user is frustrated based on the content of their voice instructions.

[1668] Data storage and sharing

[1669] The audio and video data collected from users is stored on a cloud server (e.g., AWS, Google Cloud). PostgreSQL or MongoDB is used as the database. The data is managed in a way that makes it properly categorized and searchable.

[1670] Exception handling assistance

[1671] If the emotion engine detects negative emotions from the user, the system will provide guidance for immediate action and automatically initiate assistance through technical staff if support is needed.

[1672] Specific examples

[1673] For example, in a scenario where a worker is explaining the production procedure for a new part in a factory, the worker might give the robot a voice command such as "Explain the production procedure for the new part." In this case:

[1674] 1. The device analyzes the voice commands and converts them into text.

[1675] 2. The emotion engine recognizes the user's emotions and, if it determines that the worker is feeling frustrated, it presents detailed instructions in video format.

[1676] 3. Detailed visual guides are provided to help workers understand better.

[1677] Prompt Sentence Examples

[1678] Here are some example prompts to input to a generative AI model:

[1679] text

[1680] A worker commands the robot to "explain the production procedure for a new part." The emotion engine recognizes that the worker is expressing frustration. In this case, the robot displays a detailed video of the procedure to help the worker understand.

[1681] In this way, the system provided by the present invention not only achieves data management and business process optimization for small and medium-sized enterprises, but also provides flexible support according to the emotional state of employees.

[1682] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1683] Step 1:

[1684] The user gives a voice command such as "Explain the production procedure for the new part."

[1685] Input: A spoken command from the user.

[1686] Output: Audio data.

[1687] Specific action: The user speaks into the device's microphone.

[1688] Step 2:

[1689] The terminal uses voice recognition software to convert the voice data into text.

[1690] Input: Audio data.

[1691] Output: Textual instructions.

[1692] Specific operation: Uses the SpeechRecognition library to convert voice data into text. Uses devices such as Raspberry Pi.

[1693] Step 3:

[1694] The device sends the converted text to the emotion engine's API to analyze the user's emotions.

[1695] Input: Instructions in text format.

[1696] Output: Sentiment analysis results.

[1697] Specific operation: The device sends text data to the emotion recognition API, and the emotion engine analyzes the emotion.

[1698] Step 4:

[1699] If the emotion engine detects frustration, the device will display supportive instructions.

[1700] Input: Sentiment analysis results.

[1701] Output: Instructions and procedures for support.

[1702] Specific operation: Based on the analysis results, the device displays an easy-to-understand visual guide to the user.

[1703] Step 5:

[1704] The server stores the emotion engine's analysis results and the contents of the voice instructions as a log.

[1705] Input: Sentiment analysis results and instructions in text format.

[1706] Output: Log data.

[1707] Specific operation: Works with a cloud server (e.g. AWS, Google Cloud) to store results in a database.

[1708] Step 6:

[1709] The user follows the provided visual guide as they work.

[1710] Enter: a visual guide.

[1711] Output: User's work progress.

[1712] Specific operation: The user learns and executes the production procedure for a new part by following the video and text guide displayed on the device screen.

[1713] By providing a detailed explanation of the processing performed at each step and clearly showing the overall flow of the system, the quality of the user experience can be improved and business efficiency for small and medium-sized enterprises can be improved.

[1714] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1715] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1716] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1717] [Fourth embodiment]

[1718] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1719] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1720] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1721] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1722] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1723] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1724] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1725] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1726] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1727] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1728] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1729] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1730] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1731] This invention is a generative AI platform for small and medium-sized enterprises, which supports the digitalization and productivity improvement of small and medium-sized enterprises by providing dedicated environments for each company and a common environment.

[1732] Overall system overview

[1733] The system provides each small and medium-sized enterprise with a dedicated database, safely manages confidential data for each company, and creates an environment where common expertise can be utilized. It also includes a variety of functions, such as managing audio and video data, sharing business know-how, and processing billing information.

[1734] Setting up a dedicated environment

[1735] 1. Server:

[1736] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[1737] Initialize the administrator account and provide credentials to the company.

[1738] 2. User (Administrator):

[1739] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[1740] 3. Server:

[1741] Manage employee account credentials and implement security measures to prevent unauthorized access.

[1742] Data entry and management

[1743] 1. User:

[1744] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[1745] Use a specific data entry form and click the "Save" button.

[1746] 2. Terminal:

[1747] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[1748] 3. Server:

[1749] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[1750] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[1751] Leveraging expertise in a common environment

[1752] 1. User:

[1753] Enter your tax and legal questions in a common environment and click the "Submit" button.

[1754] 2. Terminal:

[1755] The user's question is sent to a common environment and the relevant information is searched for.

[1756] 3. Server:

[1757] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[1758] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[1759] Audio and video data management

[1760] 1. User:

[1761] Record and upload business content and know-how in audio or video format.

[1762] 2. Terminal:

[1763] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[1764] 3. Server:

[1765] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[1766] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[1767] Manage your billing information

[1768] 1. Server:

[1769] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[1770] 2. Terminal:

[1771] Display billing information to administrators and provide payment links.

[1772] 3. Admin user:

[1773] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[1774] Example: At the end of each month, the server aggregates the usage history and generates the appropriate invoice, which the administrator can then view and easily complete the payment.

[1775] In this way, the present invention combines data management for each small and medium-sized enterprise with the utilization of common expertise to achieve efficient business operations and accurate billing processing using voice and video, thereby resolving many of the challenges faced by small and medium-sized enterprises and improving productivity.

[1776] The processing flow will be explained below.

[1777] Setting up a dedicated environment

[1778] Step 1:

[1779] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[1780] Step 2:

[1781] The server initializes the administrator account and provides the credentials to the company.

[1782] Step 3:

[1783] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[1784] Step 4:

[1785] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[1786] Data entry and management

[1787] Step 1:

[1788] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[1789] Step 2:

[1790] A user uses a particular data entry form and clicks the "Save" button.

[1791] Step 3:

[1792] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[1793] Step 4:

[1794] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[1795] Leveraging expertise in a common environment

[1796] Step 1:

[1797] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[1798] Step 2:

[1799] The terminal transmits the user's question to the common environment and searches for relevant information.

[1800] Step 3:

[1801] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[1802] Audio and video data management

[1803] Step 1:

[1804] Users record and upload their business operations and know-how in audio and video formats.

[1805] Step 2:

[1806] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[1807] Step 3:

[1808] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[1809] Manage your billing information

[1810] Step 1:

[1811] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[1812] Step 2:

[1813] The server generates an invoice and notifies the administrative user.

[1814] Step 3:

[1815] The terminal displays billing information to the administrator and provides a payment link.

[1816] Step 4:

[1817] Admin user reviews invoices and makes payments online.

[1818] Step 5:

[1819] The server updates the records in the system when the payment is completed.

[1820] Example 1

[1821] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1822] Currently, many small and medium-sized enterprises have not made much progress in digitalization, and are facing challenges in business efficiency and productivity. Furthermore, it is difficult to utilize the different data management and expertise that each company has, and in particular, the management of audio and video data and proper billing processing are insufficient. As a result, they are unable to obtain real-time information or appropriate advice for daily operations and management decisions, which is hindering productivity improvement.

[1823] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1824] In this invention, the server includes a means for generating a database for storing data specific to each company, a means for managing authentication information for each company, a means for inputting, saving, and managing confidential data for each company, a means for accessing a common knowledge base and providing related information, a means for uploading, managing, and analyzing audio and video data, and a means for generating and managing billing information based on each company's usage history. This allows small and medium-sized enterprises to promote digitalization, obtain information in real time, and receive appropriate advice. It also improves business efficiency and accurate billing processing, significantly increasing productivity.

[1825] A "database" is a system for managing, storing, and efficiently manipulating a collection of data created specifically for each company.

[1826] "Authentication information" refers to information required for a user to access a system, and is primarily composed of a username, password, security token, and the like.

[1827] "Confidential data" refers to data that should only be used within a specific company, such as a company's sales data or customer information, and must be managed with high security.

[1828] A "knowledge base" is a database in which various specialized knowledge and information are organized and stored, and is used to provide appropriate answers to user questions.

[1829] "Audio data" is a recording of a user's work content or know-how, and is mainly saved in the form of an audio file.

[1830] "Video data" refers to work content or know-how recorded by a user, and is mainly saved in the form of a video file.

[1831] "Uploading" is the process by which a user sends data from their device to a server and stores it.

[1832] "Analysis" is the process of analyzing uploaded audio or video, converting the content into text, and generating metadata.

[1833] "Metadata" is auxiliary information added to facilitate data management and search, and is used for classification and search.

[1834] "Billing information" is data including the billing amount calculated based on the usage record of each company and its details, and is managed as a bill.

[1835] This invention relates to a generative AI platform for small and medium-sized enterprises, which supports their digitalization and productivity improvement by providing dedicated and shared environments for each company. This platform integrates a dedicated database, secure authentication management, confidential data processing, utilization of specialized knowledge, audio and video data management, and proper processing of billing information.

[1836] Setting up a dedicated environment

[1837] The server creates a dedicated database for each small or medium-sized business, allowing data to be kept separate and preventing it from being mixed with other companies' data. The database contains tables for sales, customers, invoices, etc., and can be managed exclusively by each company.

[1838] The server then initializes an administrator account and sends the credentials to the company, using the bcrypt library for password hashing. After receiving the administrator account, the company administrator can add employee accounts and set the necessary permissions after the first login.

[1839] Additionally, the server uses an ID management service such as Auth0 to securely manage authentication information and performs authentication using JWT tokens, implementing advanced security measures to prevent unauthorized access.

[1840] Data entry and management

[1841] Users log in to the company's dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form, and by clicking the "Save" button, the data is sent to the server.

[1842] The device sends the entered data to a company-specific API endpoint and makes a save request to the server. This data is sent using HTTPS, ensuring security. The server receives the data, validates it, and saves it in a MySQL database. The user is notified when the save is successful.

[1843] Leveraging expertise in a common environment

[1844] Users access the common environment and enter their tax or legal questions into a text form, enter a specific question such as "How do I file my taxes the latest?", and click the "Submit" button.

[1845] The device sends the question to the common environment and generates a prompt to search for the relevant information in the knowledge base. The server retrieves the appropriate information from the knowledge base and returns it to the user, allowing the user to instantly access the latest legal and tax information.

[1846] Audio and video data management

[1847] Users can record their work details and know-how in audio and video formats. For example, they can record a video of the assembly procedure for a new product on their smartphone and upload it to the system.

[1848] The device sends the uploaded audio and video data to a company-specific storage (e.g., AWS S3), where the server converts the audio to text using AWS Transcribe and stores it as metadata in DynamoDB, making it easily searchable and browsable later.

[1849] Manage your billing information

[1850] The server aggregates the usage data for each company monthly, calculates the billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator user.

[1851] The terminal then displays the billing information to the administrator and provides an online payment link, which uses the Stripe payment gateway and automatically updates records in the system once payment is completed. Admin users can view their invoices and quickly pay online.

[1852] Prompt Sentence Examples

[1853] 1. "How do I enter new sales data?"

[1854] 2. "How do I file my taxes the latest?"

[1855] By implementing these procedures, the present invention solves many of the problems faced by small and medium-sized enterprises, and improves business efficiency and productivity.

[1856] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1857] Setting up a dedicated environment

[1858] Step 1:

[1859] The server creates a database for each small or medium-sized business. Specifically, it uses a MySQL database with a different schema for each company. For example, for Company A, it creates tables for sales, customers, invoices, etc.

[1860] Input: Small and Medium Enterprise Information

[1861] Output: Company-specific database

[1862] Step 2:

[1863] The server initializes an administrator account, hashes the credentials using the bcrypt library, and then emails the resulting credentials to the company.

[1864] Input: Administrator information

[1865] Output: Admin account with hashed credentials

[1866] Step 3:

[1867] The user (administrator) logs into the dedicated environment and adds accounts for other employees. The administrator sets the appropriate permissions for each account and grants the access rights required for work.

[1868] Input: Employee information

[1869] Output: Permissioned employee accounts

[1870] Step 4:

[1871] The server manages employee account authentication information, uses identity management services such as Auth0 to prevent unauthorized access, generates JWT tokens, and implements security measures.

[1872] Input: Employee Credentials

[1873] Output: Secure credentials

[1874] Data entry and management

[1875] Step 1:

[1876] Users log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a dedicated form and then the user clicks the "Save" button.

[1877] Input: Sales data, customer information

[1878] Output: Form data

[1879] Step 2:

[1880] The device sends the data entered by the user in JSON format to the enterprise API endpoint and makes a save request to the server. This request is sent using HTTPS, ensuring security.

[1881] Input: Form Data

[1882] Output: API request

[1883] Step 3:

[1884] The server validates the data received from the terminal and saves it in the appropriate table (sales, customer, etc.) in the dedicated database. It notifies the terminal that saving is complete.

[1885] Input: API request data

[1886] Output: Database entry, save success notification

[1887] Leveraging expertise in a common environment

[1888] Step 1:

[1889] Users access the common environment, enter a question, such as "How do I file my taxes the latest?" into a text form, and click the "Submit" button.

[1890] Input: Question

[1891] Output: Question submitted

[1892] Step 2:

[1893] The terminal transmits the transmitted question to the common environment and generates a prompt to search for the relevant information within the knowledge base.

[1894] Input: Submitted question

[1895] Output: Search prompt

[1896] Step 3:

[1897] The server retrieves relevant information from the knowledge base, generates an answer, and sends it back to the device, providing detailed guidance based on tax or legal information, for example.

[1898] Input: Search prompt

[1899] Output: Answer information

[1900] Audio and video data management

[1901] Step 1:

[1902] Users record and record their work content and know-how in audio or video format and upload the files to the system.

[1903] Input: Audio files, video files

[1904] Output: Upload request

[1905] Step 2:

[1906] The device sends the uploaded audio and video data to the company's dedicated storage and makes a storage request to the server, for example, AWS S3, to store the data securely.

[1907] Input: Upload request

[1908] Output: Save request to storage

[1909] Step 3:

[1910] The server converts the audio to text using AWS Transcribe and stores the text data along with metadata in DynamoDB, which makes it easier to search and categorize later.

[1911] Input: Storage data

[1912] Output: Text data, metadata

[1913] Manage your billing information

[1914] Step 1:

[1915] The server aggregates each company's monthly usage data, calculates the appropriate billing amount, and then uses a Python script to generate a PDF invoice and email it to the administrator.

[1916] Input: Monthly usage record

[1917] Output: Invoice PDF

[1918] Step 2:

[1919] The terminal displays billing information to the administrator and provides an online payment link, using the Stripe payment gateway.

[1920] Input: Invoice PDF, administrator information

[1921] Output: Payment link

[1922] Step 3:

[1923] The admin user reviews the invoice and clicks on the provided link to complete the payment online. Once the payment is successful, the transaction record is automatically updated in the system.

[1924] Input: Payment information

[1925] Output: Updated transaction record

[1926] (Application example 1)

[1927] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1928] In recent years, the slow pace of digitalization among small and medium-sized enterprises has become a problem. While there is a demand for improved productivity, technical resources are often lacking. In particular, for companies that use automated vehicles, managing and optimizing vehicle operation data requires a great deal of time and effort, which leads to a decline in business efficiency. In addition, it is difficult to safely manage confidential data and share specialized knowledge, and audio and video data are not being effectively utilized. There is a need for a system that can solve these issues and support the digitalization and productivity improvement of small and medium-sized enterprises.

[1929] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1930] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for collecting operation data of automatically controlled vehicles and transmitting it to a company-dedicated server, and means for analyzing the operation data and optimizing operation plans in real time. This enables efficient management and optimization of operation data of automatically controlled vehicles, enabling digitalization and improved productivity for small and medium-sized enterprises.

[1931] A "database" is a system for systematically organizing, storing, and managing information.

[1932] "Authentication information" is information used to verify the authenticity of a user or system.

[1933] "Confidential data" refers to important data that could cause harm if leaked to the outside.

[1934] A "knowledge base" is a database that systematically compiles specific knowledge and information.

[1935] "Audio data" means data that is a digital representation of a recording of spoken words or other sounds.

[1936] "Moving image data" is digital data that expresses movement by playing back a series of still images at regular time intervals.

[1937] "Billing information" is information indicating the cost for providing a service or product.

[1938] An "automated vehicle" is a vehicle that can operate automatically without human operation.

[1939] "Operation data" refers to various data related to operation, such as the vehicle's position, speed, and condition.

[1940] A "server" is a computer system that provides services over a network.

[1941] "Real-time" refers to a state in which information and processing are carried out immediately without delay.

[1942] This invention is a system that collects and manages operational data from autonomous vehicles to support the digitalization and productivity improvement of small and medium-sized enterprises. Specifically, it creates a dedicated database for each company, securely manages authentication information and confidential data, and provides an environment where a common knowledge base can be easily accessed. The system also includes functions such as uploading and managing audio and video data, generating and managing billing information for each company, and collecting, analyzing, and optimizing operational data from autonomous vehicles.

[1943] System Configuration

[1944] 1. Creation and management of a dedicated database for each company

[1945] The server creates a dedicated database for each company and stores confidential data in that database. Each company's authentication information is managed in this database, and security measures are implemented to prevent unauthorized access. For example, an administrator completes the initial login and adds employee accounts, setting appropriate permissions.

[1946] 2. Access to a common, dedicated knowledge base

[1947] Users can use a common, dedicated knowledge base to ask tax and legal questions and get answers in real time, improving work efficiency. For example, by entering a question like "What is the latest tax filing method?" into the system, the latest information is retrieved from the common environment.

[1948] 3. Audio and video data management

[1949] Users record their work and know-how in audio or video format and upload it to the system. The server analyzes the audio data, converts it into text, and generates metadata, which makes it easy for other employees to search and refer to. For example, a video of the assembly procedure for a new product can be recorded and made available to each employee.

[1950] 4. Management of operational data for autonomous vehicles

[1951] Operational data from autonomous vehicles is sent in real time from the vehicle's terminal to a company-dedicated server. This data includes location information, battery level, and operating status. The server analyzes this data and optimizes operation plans in real time. For example, if the battery level is low, it will guide the vehicle to the nearest charging station.

[1952] 5. Generative AI Model and Prompts

[1953] Generative AI models are used to perform various data analyses and optimizations. For example, a prompt such as "Please tell me how to develop a system that manages autonomous vehicle operation data in real time and provides optimal operation plans" can be generated and input into the AI ​​model to achieve advanced analysis.

[1954] Hardware and Software Examples

[1955] Hardware: Servers, smartphones, autonomous vehicle terminals

[1956] Software: Python, requests library, enterprise-specific API endpoints

[1957] The above configuration and means will enable digitalization and productivity improvement in small and medium-sized enterprises.

[1958] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1959] Step 1:

[1960] The user logs into the dedicated environment and inputs vehicle operation data. Specifically, the user enters data such as the vehicle's location, battery level, and operating status into an input form and clicks the "Submit" button. The input data also includes the vehicle's identification information.

[1961] Step 2:

[1962] The device sends the operational data entered by the user to a company-specific API endpoint, where it is packaged in JSON format and sent as an HTTP POST request to the API endpoint. The device processes the data and sends it to the server via a communication protocol.

[1963] Step 3:

[1964] The server analyzes the operational data received from the terminal and stores it in the company's dedicated database. The data is converted into the appropriate format and inserted into the specified table in the database. Once the data has been saved, the server notifies the terminal that the data has been saved successfully.

[1965] Step 4:

[1966] The server performs real-time analysis based on the stored operational data to optimize the operation plan. This analysis involves statistically processing multidimensional data such as location information, battery level, and operation status to select the optimal operation route and charging station. Generative AI models are utilized here.

[1967] Step 5:

[1968] The server sends the analysis results back to the device via a dedicated API endpoint. Specifically, it includes an optimized operation plan and instructions for when emergency charging is required. The device receives this analysis data and notifies the user.

[1969] Step 6:

[1970] The user checks the analysis results notified from the device and modifies the operation plan as necessary. For example, if an instruction to go to a charging station is issued, the user will act promptly in accordance with the instruction. The input in this step is the analysis results, and the output is the user's action.

[1971] The above processing steps seamlessly realize a series of steps including data collection, analysis, optimization, and notification, enabling small and medium-sized enterprises to digitalize and improve productivity.

[1972] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1973] This invention combines a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions to provide a more personalized user experience and promote business efficiency.

[1974] Overall system overview

[1975] The system provides each small and medium-sized enterprise with a dedicated database, safely managing each company's confidential data and creating an environment where they can utilize shared expertise. It also includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[1976] Setting up a dedicated environment

[1977] 1. Server:

[1978] We create a dedicated database for each small and medium-sized enterprise and provide an environment where corporate data can be safely stored.

[1979] Initialize the administrator account and provide credentials to the company.

[1980] 2. User (Administrator):

[1981] After logging in for the first time, add accounts for other employees and set the appropriate permissions for each account.

[1982] 3. Server:

[1983] Manage employee account credentials and implement security measures to prevent unauthorized access.

[1984] Data entry and management

[1985] 1. User:

[1986] Log in to a dedicated environment and enter confidential data such as sales data and customer information.

[1987] Use a specific data entry form and click the "Save" button.

[1988] 2. Terminal:

[1989] The entered data is sent to a company-specific API endpoint and a save request is sent to the server.

[1990] 3. Server:

[1991] The data received from the device is stored in a company-specific database, and the device is notified that the data has been saved.

[1992] Example: Users enter daily sales data and the system stores this data in real time, allowing them to instantly understand sales trends and improve business operations.

[1993] Leveraging expertise in a common environment

[1994] 1. User:

[1995] Enter your tax and legal questions in a common environment and click the "Submit" button.

[1996] 2. Terminal:

[1997] The user's question is sent to a common environment and the relevant information is searched for.

[1998] 3. Server:

[1999] It retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[2000] Example: When a user asks "How to file taxes the latest?" the system retrieves the latest information from the common environment and provides detailed guidance.

[2001] Audio and video data management

[2002] 1. User:

[2003] Record and upload business content and know-how in audio or video format.

[2004] 2. Terminal:

[2005] The recording and video data is sent to the company's dedicated storage and a storage request is sent to the server.

[2006] 3. Server:

[2007] It analyzes uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[2008] Example: A user can record a video of the assembly process for a new product, and the system can categorize the video, making it easily accessible to other employees.

[2009] Manage your billing information

[2010] 1. Server:

[2011] Summarize monthly usage data, calculate billing amounts for each company, generate invoices, and notify administrator users.

[2012] 2. Terminal:

[2013] Display billing information to administrators and provide payment links.

[2014] 3. Admin user:

[2015] View your invoice and pay online, and your payment will be updated in the system once it is complete.

[2016] Utilizing the Emotion Engine

[2017] 1. User:

[2018] Interact with the system through voice and text.

[2019] 2. Emotion Engine:

[2020] Analyzes the user's voice and text to recognize emotions.

[2021] 3. Server:

[2022] It tailors its response based on the perceived emotion and provides personalized feedback to the user.

[2023] 4. Terminal:

[2024] The tailored response is displayed to the user, improving the user's experience.

[2025] Example: If a user gives a speech input that indicates frustration to the system, the emotion engine will recognize that emotion and the system will provide a more supportive response.

[2026] Accumulation and use of emotional data

[2027] 1. Emotion Engine:

[2028] Emotional data for each user is accumulated and saved as a history.

[2029] 2. Server:

[2030] The accumulated emotional data is analyzed to extract trends and patterns for individual users.

[2031] 3. Terminal:

[2032] The analysis results are used to further personalize the user experience and provide adaptive feedback.

[2033] Example: If a user regularly expresses negative sentiment towards the same question, the system can take action to improve the answer to that question and increase user satisfaction.

[2034] In this way, the present invention combines data management for each SME, utilization of common expertise, and an emotion engine to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[2035] The processing flow will be explained below.

[2036] Setting up a dedicated environment

[2037] Step 1:

[2038] The server creates a dedicated database for each small and medium-sized business, providing an environment where corporate data can be safely stored.

[2039] Step 2:

[2040] The server initializes the administrator account and provides the credentials to the company.

[2041] Step 3:

[2042] After logging in for the first time, the user (administrator) adds accounts for other employees and sets appropriate permissions for each account.

[2043] Step 4:

[2044] The server manages employee account authentication information and implements security measures to prevent unauthorized access.

[2045] Data entry and management

[2046] Step 1:

[2047] Users log into a dedicated environment and enter sensitive data such as sales data or customer information.

[2048] Step 2:

[2049] A user uses a particular data entry form and clicks the "Save" button.

[2050] Step 3:

[2051] The device sends the entered data to a company-specific API endpoint and sends a save request to the server.

[2052] Step 4:

[2053] The server stores the data received from the terminal in a company-specific database and notifies the terminal that the data has been saved.

[2054] Leveraging expertise in a common environment

[2055] Step 1:

[2056] A user enters their tax or legal question in a common environment and clicks the "Submit" button.

[2057] Step 2:

[2058] The terminal transmits the user's question to the common environment and searches for relevant information.

[2059] Step 3:

[2060] The server retrieves relevant information from the common environment knowledge base and returns it to the terminal.

[2061] Audio and video data management

[2062] Step 1:

[2063] Users record and upload their business operations and know-how in audio and video formats.

[2064] Step 2:

[2065] The device sends the recorded and video data to the company's dedicated storage and sends a save request to the server.

[2066] Step 3:

[2067] The server analyzes the uploaded audio data, converts it into text, and generates metadata to make the data categorized and searchable.

[2068] Manage your billing information

[2069] Step 1:

[2070] The server compiles usage data on a monthly basis and calculates the billing amount for each company.

[2071] Step 2:

[2072] The server generates an invoice and notifies the administrative user.

[2073] Step 3:

[2074] The terminal displays billing information to the administrator and provides a payment link.

[2075] Step 4:

[2076] Admin user reviews invoices and makes payments online.

[2077] Step 5:

[2078] The server updates the records in the system when the payment is completed.

[2079] Utilizing the Emotion Engine

[2080] Step 1:

[2081] The user interacts with the system through voice and text.

[2082] Step 2:

[2083] The emotion engine analyzes the user's voice and text to recognize emotions.

[2084] Step 3:

[2085] The server tailors the response based on the recognized emotion and provides personalized feedback to the user.

[2086] Step 4:

[2087] The terminal displays the adjusted response to the user, improving the user experience.

[2088] Accumulation and use of emotional data

[2089] Step 1:

[2090] The emotion engine accumulates each user's emotion data and saves it as a history.

[2091] Step 2:

[2092] The server analyzes the accumulated emotional data and extracts trends and patterns for individual users.

[2093] Step 3:

[2094] The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[2095] Example 2

[2096] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2097] Small and medium-sized enterprises (SMEs) have a wide range of tasks, including data management, utilizing specialized knowledge, and managing billing information, making it difficult to perform these tasks efficiently. Furthermore, they lack a system that can properly recognize employee emotions and provide personalized feedback. Therefore, there is a need for a system that can improve both work efficiency and employee satisfaction.

[2098] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2099] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for saving and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for recognizing user emotions, accumulating and analyzing emotion data, and means for adjusting responses and providing personalized feedback based on the recognized emotions, thereby enabling small and medium-sized enterprises to efficiently manage data and utilize knowledge, and to provide personalized feedback according to user emotions.

[2100] "Company-specific database" refers to an individual database set up by each company to securely store and manage its own data.

[2101] "Authentication Information" means a username, password, or other authentication means to grant access to a System.

[2102] "Confidential data" refers to important information held within a company, such as sales data, customer information, and business know-how.

[2103] A "specialized knowledge base" refers to a database that compiles information and guidelines on a specific field.

[2104] "Means for uploading and managing audio and video" refers to the functionality that allows users to upload audio and video data to the system and store and manage it.

[2105] "Billing Information" refers to the fees calculated based on each company's service usage record and information related to the payment thereof.

[2106] "Means for recognizing emotions and storing and analyzing emotional data" refers to the function of analyzing emotions from the user's voice and text, and collecting, saving, and analyzing that data.

[2107] "Means for adjusting responses and providing personalized feedback based on recognized emotions" refers to the ability of the system to dynamically generate responses and provide individualized feedback based on the user's emotions.

[2108] This invention is a system that provides a more personalized user experience and promotes business efficiency by combining a generative AI platform for small and medium-sized enterprises with an emotion engine that recognizes user emotions. This system includes the following elements.

[2109] Overall system overview

[2110] The system provides each small and medium-sized enterprise with a dedicated database to securely manage confidential data for each company. It also accesses a common dedicated knowledge base and includes a variety of functions, such as managing audio and video data, recognizing user emotions, sharing business know-how, and processing billing information.

[2111] Setting up a dedicated environment

[2112] 1. Server:

[2113] We create a dedicated database for each small and medium-sized business and provide an environment where corporate data can be safely stored, using database systems such as MySQL and PostgreSQL.

[2114] Initialize an administrator account and provide credentials to the company, which are managed using a framework such as Django or Ruby on Rails.

[2115] 2. User (Administrator):

[2116] After logging in for the first time, you can add accounts for other employees and set the appropriate permissions for each account. Administrative functions are performed through the web interface.

[2117] 3. Server:

[2118] Employee account authentication information is managed using security services such as Auth0 and Firebase, and security measures are implemented to prevent unauthorized access.

[2119] Data entry and management

[2120] 1. User:

[2121] Log in to a dedicated environment and enter confidential data such as sales data and customer information. The data is entered using a data entry form built on a front-end framework (e.g., React, Vue.js).

[2122] 2. Terminal:

[2123] The entered data is sent to a company-specific RESTful API endpoint, and a save request is sent to the server. The communication protocol is HTTP.

[2124] 3. Server:

[2125] Save the data received from the device in a company-specific database and notify the device when the save is complete. Use a backend framework (e.g., Express.js, Spring Boot).

[2126] As a specific example of how it works, users can input daily sales data and the system will save this data in real time, allowing them to instantly grasp sales trends and helping to improve business operations.

[2127] Leveraging expertise in a common environment

[2128] 1. User:

[2129] Enter your tax and legal questions in a common environment and click the "Submit" button.

[2130] 2. Terminal:

[2131] The user's question is sent to a common environment and the relevant information is searched for.

[2132] 3. Server:

[2133] It retrieves relevant information from a dedicated knowledge base in the common environment and sends it back to the device, using a wiki or other information resource.

[2134] As a specific example of how it works, when a user asks "How to file tax returns the latest?", the system retrieves the latest information from the common environment and provides detailed guidance.

[2135] Audio and video data management

[2136] 1. User:

[2137] Record and upload business content and know-how in audio or video format.

[2138] 2. Terminal:

[2139] The recording and video data is sent to the company's dedicated cloud storage (e.g., Amazon S3) and a storage request is sent to the server.

[2140] 3. Server:

[2141] The uploaded audio data is converted to text using the Google Cloud Speech-to-Text API, and metadata is generated and indexed in Elasticsearch.

[2142] As a specific example of how it works, a user can record a video of the assembly procedure for a new product, and the system will analyze the video, making it easy for other employees to search and refer to.

[2143] Manage your billing information

[2144] 1. Server:

[2145] Calculate the billing amount for each company based on monthly usage. Generate an invoice using Stripe API and notify the administrator user.

[2146] 2. Terminal:

[2147] Display billing information to administrators and provide payment links.

[2148] 3. Admin user:

[2149] View your invoice and pay online. Once payment is complete, your records will be automatically updated in the system.

[2150] Utilizing the Emotion Engine and Response Adjustment

[2151] 1. User:

[2152] Interact with the system via voice and text.

[2153] 2. Emotion Engine:

[2154] It uses IBM Watson's Natural Language Understanding API to analyze the user's voice and text and recognize emotions.

[2155] 3. Server:

[2156] It tailors responses based on perceived emotions and provides personalized feedback using generative AI models.

[2157] 4. Terminal:

[2158] The tailored response is displayed to the user, improving the user's experience.

[2159] As a specific example of how this works, if a user makes a speech input to the system that indicates frustration, the emotion engine will recognize the emotion and the system will generate and display a more supportive and understanding response.

[2160] Accumulation and analysis of emotional data

[2161] 1. Emotion Engine:

[2162] Emotion data for each user is periodically accumulated.

[2163] 2. Server:

[2164] The accumulated emotional data is analyzed using machine learning models (e.g., TensorFlow, PyTorch) to extract trends and patterns for individual users.

[2165] 3. Terminal:

[2166] Based on the analysis results, the user experience can be further personalized and adaptive feedback can be provided.

[2167] As a specific example of how it works, if a user repeatedly has a negative reaction to a particular question, that emotional data is accumulated and analyzed, and the system improves the answer to that question.

[2168] An example prompt might be, "What is the latest information on tax filing? I currently run a small business and need advice on filing my 2023 taxes."

[2169] As described above, the present invention combines data management for each SME with the utilization of common expertise, as well as an emotion engine, to achieve personalized responses and an improved user experience, thereby resolving many of the challenges faced by SMEs and improving productivity.

[2170] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2171] Step 1: User authentication

[2172] Input: The user enters their username and password on the login screen.

[2173] Processing: The device sends the input information to the Auth0 API. Auth0 verifies the authentication information and generates an authentication token.

[2174] Output: The server receives the authentication token and returns it to the device indicating that the user has been authenticated.

[2175] What it does: When a user enters their credentials on the login screen, the system verifies them through Auth0 and redirects them to the dashboard if authentication is successful.

[2176] Step 2: Enter and save data

[2177] Input: Users enter sensitive data, such as sales data or customer information, into data entry forms.

[2178] Processing: The device sends the input data to a company-specific server using a RESTful API, which stores the data in a MySQL database.

[2179] Output: The server notifies the device that the save is complete.

[2180] Specific behavior: When a user enters sales data into a React-based data entry form and clicks the "Save" button, the data is sent to the server and saved in the database.

[2181] Step 3: Find expertise

[2182] Input: User enters tax or legal question in a common environment and clicks "Submit" button.

[2183] Processing: The device sends the query to the common environment search API to search for related information.

[2184] Output: The server retrieves the relevant information from the knowledge base and returns the search results to the terminal.

[2185] What it does: When a user types "latest tax filing method" into the search bar and hits the submit button, the system displays the latest relevant tax guidelines.

[2186] Step 4: Managing audio and video data

[2187] Input: Users record and record their work content and know-how in audio or video format and upload it to the system.

[2188] Processing: The device sends the audio and video data to the company's dedicated cloud storage and sends a storage request to the server. The server analyzes the audio data and converts it into text.

[2189] Output: The server generates the parsed text and metadata and indexes it into Elasticsearch.

[2190] Specific operation: When a user records a video of their business know-how and uploads it to the system, the audio is automatically converted into text and saved in a searchable format.

[2191] Step 5: Manage your billing information

[2192] Input: The server generates billing information based on monthly usage data.

[2193] Processing: The server uses the Stripe API to generate invoices for each company and notify the admin user.

[2194] Output: The terminal displays the billing information to the administrator and provides a payment link.

[2195] Specific operation: At the end of each month, the system will compile usage data, automatically generate an invoice through Stripe, and notify the administrator user. The administrator can then log in, check the invoice, and pay online.

[2196] Step 6: Leverage your emotional engine and adjust your response

[2197] Input: The user interacts with the system through voice and text.

[2198] Processing: The emotion engine uses IBM Watson APIs to analyze the user's emotions. The server tailors responses based on the recognized emotions and provides personalized feedback using generative AI models.

[2199] Output: The device displays the adjusted response to the user, improving the user's experience.

[2200] How it works: When a user inputs speech that expresses dissatisfaction with the system, the emotion engine recognizes the emotion and the system provides a supportive response.

[2201] Step 7: Collect and analyze emotional data

[2202] Input: The emotion engine periodically collects emotion data for each user.

[2203] Processing: The server analyzes the accumulated emotional data using machine learning models to extract trends and patterns of the user's emotions.

[2204] Output: The device uses the analysis results to further personalize the user experience and provide adaptive feedback.

[2205] What it does: If a user expresses negative sentiment towards a particular question, that data is stored in the system to improve future answers.

[2206] (Application example 2)

[2207] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2208] The purpose of this invention is to improve business efficiency and productivity in small and medium-sized enterprises. Previous systems did not adequately utilize company-specific data management and specialized knowledge, making it difficult to optimize business processes. Furthermore, there was a lack of personalized responses and support that took employee emotions into consideration, leaving the issue of improving the working environment unresolved.

[2209] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2210] In this invention, the server includes means for creating a database dedicated to each company, means for managing authentication information for each company, means for storing and managing confidential data for each company, means for accessing a common dedicated knowledge base, means for uploading and managing audio and video, means for generating and managing billing information for each company, means for interpreting voice instructions and dynamically changing work plans, and means for recognizing user emotions using an emotion engine and adjusting responses, thereby enabling more efficient data management, optimization of business processes, and personalized support based on employee emotions.

[2211] A "company-specific database" is a database in which data is securely stored and managed in a format that each company can use independently.

[2212] "Authentication information" is identification information required when a user accesses a system, and is information used to ensure security.

[2213] "Confidential data" refers to important information held by a company that must not be leaked to the outside.

[2214] A "dedicated knowledge base" is a database in which specialized knowledge and information are accumulated and managed, and can be shared when necessary.

[2215] "Audio and video uploading and management" refers to the process by which users provide audio and video data to the system, which then stores and organizes that data.

[2216] "Billing information" refers to information regarding fees and payments calculated based on usage history.

[2217] "Means for interpreting voice instructions and dynamically changing work plans" refers to a function that understands instructions provided by voice and automatically adjusts the schedule and content of work.

[2218] An "emotion engine" is a system that analyzes the user's emotions and generates and adjusts appropriate responses based on those emotions.

[2219] The present invention aims to improve business efficiency and productivity in small and medium-sized enterprises. Specific embodiments will be described below.

[2220] Overall system configuration

[2221] The system consists of the following main elements:

[2222] Server Roles

[2223] The server creates a dedicated database for each company, manages authentication information for each company, stores and manages confidential data, accesses a common dedicated knowledge base, uploads and manages audio and video data, and generates and manages billing information for each company.

[2224] Device Role

[2225] The device provides an interface for users to issue voice commands, interprets the voice commands, dynamically changes the task plan, and uses an emotion engine to recognize the user's emotions and adjust responses accordingly.

[2226] User Roles

[2227] Users interact with the system by giving instructions to it through voice or text, for example, they can ask the system to explain a specific work procedure.

[2228] Main processing flow

[2229] Speech recognition processing

[2230] The device uses voice recognition technology to analyze user instructions, using a microphone and a Raspberry Pi to convert voice data into text using the SpeechRecognition library.

[2231] Emotion recognition processing

[2232] The emotion engine analyzes emotions from the user's voice or text using an external emotion recognition API, for example, to determine whether the user is frustrated based on the content of their voice instructions.

[2233] Data storage and sharing

[2234] The audio and video data collected from users is stored on a cloud server (e.g., AWS, Google Cloud). PostgreSQL or MongoDB is used as the database. The data is managed in a way that makes it properly categorized and searchable.

[2235] Exception handling assistance

[2236] If the emotion engine detects negative emotions from the user, the system will provide guidance for immediate action and automatically initiate assistance through technical staff if support is needed.

[2237] Specific examples

[2238] For example, in a scenario where a worker is explaining the production procedure for a new part in a factory, the worker might give the robot a voice command such as "Explain the production procedure for the new part." In this case:

[2239] 1. The device analyzes the voice commands and converts them into text.

[2240] 2. The emotion engine recognizes the user's emotions and, if it determines that the worker is feeling frustrated, it presents detailed instructions in video format.

[2241] 3. Detailed visual guides are provided to help workers understand better.

[2242] Prompt Sentence Examples

[2243] Here are some example prompts to input to a generative AI model:

[2244] text

[2245] A worker commands the robot to "explain the production procedure for a new part." The emotion engine recognizes that the worker is expressing frustration. In this case, the robot displays a detailed video of the procedure to help the worker understand.

[2246] In this way, the system provided by the present invention not only achieves data management and business process optimization for small and medium-sized enterprises, but also provides flexible support according to the emotional state of employees.

[2247] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2248] Step 1:

[2249] The user gives a voice command such as "Explain the production procedure for the new part."

[2250] Input: A spoken command from the user.

[2251] Output: Audio data.

[2252] Specific action: The user speaks into the device's microphone.

[2253] Step 2:

[2254] The terminal uses voice recognition software to convert the voice data into text.

[2255] Input: Audio data.

[2256] Output: Textual instructions.

[2257] Specific operation: Uses the SpeechRecognition library to convert voice data into text. Uses devices such as Raspberry Pi.

[2258] Step 3:

[2259] The device sends the converted text to the emotion engine's API to analyze the user's emotions.

[2260] Input: Instructions in text format.

[2261] Output: Sentiment analysis results.

[2262] Specific operation: The device sends text data to the emotion recognition API, and the emotion engine analyzes the emotion.

[2263] Step 4:

[2264] If the emotion engine detects frustration, the device will display supportive instructions.

[2265] Input: Sentiment analysis results.

[2266] Output: Instructions and procedures for support.

[2267] Specific operation: Based on the analysis results, the device displays an easy-to-understand visual guide to the user.

[2268] Step 5:

[2269] The server stores the emotion engine's analysis results and the contents of the voice instructions as a log.

[2270] Input: Sentiment analysis results and instructions in text format.

[2271] Output: Log data.

[2272] Specific operation: Works with a cloud server (e.g. AWS, Google Cloud) to store results in a database.

[2273] Step 6:

[2274] The user follows the provided visual guide as they work.

[2275] Enter: a visual guide.

[2276] Output: User's work progress.

[2277] Specific operation: The user learns and executes the production procedure for a new part by following the video and text guide displayed on the device screen.

[2278] By providing a detailed explanation of the processing performed at each step and clearly showing the overall flow of the system, the quality of the user experience can be improved and business efficiency for small and medium-sized enterprises can be improved.

[2279] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2280] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2281] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2282] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2283] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2284] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2285] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2286] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2287] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2288] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2289] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2290] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2291] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2292] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2293] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2294] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2295] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2296] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2297] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2298] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2299] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2300] The following is further disclosed regarding the above embodiment.

[2301] (Claim 1)

[2302] A means of creating a database dedicated to each company;

[2303] A means of managing each company's authentication information;

[2304] A means of storing and managing confidential data for each company,

[2305] a means for accessing a common dedicated knowledge base;

[2306] A means to upload and manage audio and video;

[2307] a means for generating and managing billing information for each business;

[2308] A system including:

[2309] (Claim 2)

[2310] 2. The system of claim 1, further comprising means for accepting user queries against a common dedicated knowledge base and providing relevant information.

[2311] (Claim 3)

[2312] 2. The system according to claim 1, further comprising means for converting uploaded voice data into text and generating metadata for classification and search.

[2313] "Example 1"

[2314] (Claim 1)

[2315] A means for generating a database to hold data specific to each company;

[2316] A means of managing each company's authentication information;

[2317] A means to input, store and manage each company's confidential data;

[2318] a means of accessing a common knowledge base and providing relevant information;

[2319] means for uploading, managing and analyzing audio and video data;

[2320] a means for generating and managing billing information based on usage history for each company;

[2321] A system including:

[2322] (Claim 2)

[2323] 2. The system according to claim 1, further comprising means for accepting a user's question to a common knowledge base and providing relevant information.

[2324] (Claim 3)

[2325] 10. The system of claim 1, further comprising means for converting uploaded audio data into text and generating metadata for classification and search.

[2326] "Application Example 1"

[2327] (Claim 1)

[2328] A means of creating a database dedicated to each company;

[2329] A means of managing each company's authentication information;

[2330] A means of storing and managing confidential data for each company,

[2331] a means for accessing a common dedicated knowledge base;

[2332] A means to upload and manage audio and video;

[2333] a means for generating and managing billing information for each business;

[2334] A means for collecting operational data of the automatically controlled vehicle and transmitting it to a company-dedicated server;

[2335] A means to analyze operation data and optimize operation plans in real time;

[2336] A system including:

[2337] (Claim 2)

[2338] 2. The system of claim 1, further comprising means for accepting user queries against the common dedicated knowledge base and providing relevant information.

[2339] (Claim 3)

[2340] 2. The system according to claim 1, further comprising means for converting uploaded voice data into text and generating metadata for classification and search.

[2341] "Example 2: Combining Emotion Engines"

[2342] (Claim 1)

[2343] A means of creating a database dedicated to each company;

[2344] A means of managing each company's authentication information;

[2345] A means of storing and managing confidential data for each company,

[2346] a means for accessing a common dedicated knowledge base;

[2347] A means to upload and manage audio and video;

[2348] a means for generating and managing billing information for each business;

[2349] means for recognizing user emotions and accumulating and analyzing emotion data;

[2350] a means for tailoring responses based on perceived emotions and providing personalized feedback;

[2351] A system including:

[2352] (Claim 2)

[2353] 2. The system of claim 1, further comprising means for accepting user queries against a common dedicated knowledge base and providing relevant information.

[2354] (Claim 3)

[2355] 2. The system according to claim 1, further comprising means for converting uploaded voice data into text and generating metadata for classification and search.

[2356] "Application example 2 when combining emotion engines"

[2357] (Claim 1)

[2358] A means of creating a database dedicated to each company;

[2359] A means of managing each company's authentication information;

[2360] A means of storing and managing confidential data for each company,

[2361] a means for accessing a common dedicated knowledge base;

[2362] A means to upload and manage audio and video;

[2363] a means for generating and managing billing information for each business;

[2364] a means for interpreting voice instructions and dynamically changing the work plan;

[2365] means for recognizing a user's emotions and adjusting a response using an emotion engine;

[2366] A system including:

[2367] (Claim 2)

[2368] 2. The system of claim 1, further comprising means for accepting user queries against a common dedicated knowledge base and providing relevant information.

[2369] (Claim 3)

[2370] 2. The system according to claim 1, further comprising means for converting uploaded voice data into text and generating metadata for classification and search. [Explanation of symbols]

[2371] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of creating a database dedicated to each company; A means of managing each company's authentication information; A means of storing and managing confidential data for each company, a means for accessing a common dedicated knowledge base; A means to upload and manage audio and video; a means for generating and managing billing information for each business; A system including:

2. 2. The system of claim 1, further comprising means for accepting user queries against a common dedicated knowledge base and providing relevant information.

3. 2. The system according to claim 1, further comprising means for converting the uploaded voice data into text and generating metadata for classification and search.

Images