System

Abstract

A system is provided.SOLUTION: A system comprising a data processing apparatus, the data processing apparatus comprising: means for receiving a user-specified dataset; means for using a generative model to convert the dataset into a particular data format; and means for providing the converted dataset to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Data processing using conventional calculation software requires many steps and time, and has the problem of being particularly cumbersome when converting data into specific data formats or inserting line breaks to improve the readability of program code. The present invention aims to provide a system for efficiently performing these tasks, thereby significantly reducing the user's time and effort. [Means for solving the problem]

[0005] The present invention provides a system including a data processing device. The data processing device includes means for receiving a dataset specified by a user, means for using a generative model to convert the dataset into a specific data format, and means for providing the converted dataset to the user. The data processing device may also include means for using a generative model to insert line breaks at appropriate positions in input program code, and may further include means for acquiring data from a clipboard. In this way, a user can efficiently convert data and improve the readability of program code.

[0006] A "data processing device" is a device or system that receives, processes, transforms, and displays user-specified data sets.

[0007] A "specified data set" is a collection of data selected or entered by a user.

[0008] A "generative model" is a model that processes and transforms data using specific algorithms and machine learning techniques.

[0009] The "receiving means" is an interface or function that allows the data processing device to acquire data from the outside.

[0010] A "means for converting" is a function or process by which the data processing device converts data received into another format.

[0011] The "means for providing" is a function for displaying or outputting the processed and converted data to the user.

[0012] "Program code" means a sequence of computer-executable instructions to perform a particular task.

[0013] "Inserting a line break" is an operation to embed a line break character at a specific position in program code to improve readability.

[0014] The "clipboard" is a memory area for temporarily storing data, and is primarily used for copy and paste operations. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention provides a system that improves the efficiency of data processing tasks performed by users through calculation software. Specifically, it provides technology that automates data format conversion and line break insertion in program code, focusing on data processing devices.

[0037] composition

[0038] This system consists of a user terminal, a server, and a generative model. The user terminal serves as the interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques.

[0039] What the program does

[0040] Data format conversion

[0041] 1. Data selection and acquisition

[0042] The user selects the required data using calculation software (e.g. Excel), and the data is then copied.

[0043] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[0044] 2. Data transmission and conversion

[0045] The terminal transmits the acquired data to the server.

[0046] The server parses the received data and converts it into a specific SQL format using a generative model, which includes extracting column names and generating the appropriate SQL syntax.

[0047] 3. Return and display of converted data

[0048] The server returns the converted SQL data to the terminal.

[0049] The terminal displays the received SQL data to the user, who can then check it and use it as an SQL query.

[0050] Inserting a line break

[0051] 1. Enter and submit SQL code

[0052] The user types or copies and pastes the SQL code into the terminal.

[0053] The terminal sends the entered code to the server.

[0054] 2. Line break handling

[0055] The server parses the received SQL code and uses a generative model to insert line breaks at appropriate places, a process that automatically determines where to break lines to improve readability.

[0056] 3. Return and display of code after line break

[0057] The server returns the SQL code with line breaks inserted to the terminal.

[0058] The terminal displays the received SQL code with line breaks to the user, allowing the user to check the SQL code with improved readability.

[0059] Specific examples

[0060] Data format conversion example

[0061] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0062] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0063] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0064] Example of inserting a line break

[0065] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0066] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0067] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0068] The configuration and processing of the present invention allow the user to quickly and efficiently convert data into a specific data format and improve the readability of program code.

[0069] The processing flow will be explained below.

[0070] Data format conversion

[0071] Step 1:

[0072] The user selects the necessary data in calculation software (e.g., Excel). For example, the user selects the range of columns "Fiscal Year," "Sales," and "Expenses" in an Excel sheet.

[0073] Step 2:

[0074] Copies the data selected by the user. The selected data is transferred to the clipboard.

[0075] Step 3:

[0076] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[0077] Step 4:

[0078] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[0079] Step 5:

[0080] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[0081] Step 6:

[0082] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0083] Step 7:

[0084] The server sends the converted SQL data to the terminal as an HTTP response.

[0085] Step 8:

[0086] The terminal displays the received SQL data to the user in a dedicated window or message box.

[0087] Step 9:

[0088] The user checks the displayed SQL data and pastes it into a database for use.

[0089] Inserting a line break

[0090] Step 1:

[0091] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0092] Step 2:

[0093] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[0094] Step 3:

[0095] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[0096] Step 4:

[0097] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[0098] Step 5:

[0099] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[0100] Step 6:

[0101] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[0102] Step 7:

[0103] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[0104] Step 8:

[0105] The user can review the displayed SQL code and edit or use it as needed.

[0106] The above are the specific processing steps for data format conversion and line break insertion in this system.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Conventional data processing devices require users to manually convert data into a specific format, which requires time and effort. Furthermore, manually inserting line breaks to improve program code readability is cumbersome and inefficient. This reduces user work efficiency and increases the likelihood of errors.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes a means for receiving a dataset specified by a user, a means for using a generative AI model to convert the dataset into a specific data format, and a means for providing the converted dataset to the user. This allows the user to convert data formats quickly and accurately. The server also includes a means for using a generative AI model to insert line breaks at appropriate positions in input program code, and a means for returning the code after the line breaks to the user terminal. This improves the readability of the program code and further enhances the user's work efficiency.

[0112] "User" means a person or entity that uses the system to process data or manipulate program code.

[0113] A "dataset" is a set of data that is subject to processing or transformation and may contain data items in different formats.

[0114] A "generative AI model" is a collection of algorithms that use machine learning techniques to automate specific tasks (such as converting data formats or inserting line breaks into code).

[0115] "Receiving means" refers to means for transmitting a data set or program code designated by a user to the system, and includes network communication and the like.

[0116] A "transformation means" is a means that performs a procedure or process to convert a received dataset into a particular format, typically using a generative AI model.

[0117] The "provision means" is a means for presenting the converted results to the user, and is carried out through a display device or a communication device.

[0118] The "clipboard" is a temporary data storage area that the user uses for copy and paste operations.

[0119] A "server" is a computer system that receives, analyzes, transforms, and returns data and has the processing power to run a generative AI model.

[0120] A "terminal" is a computer device that is directly operated by a user, and is a device for inputting data and displaying results.

[0121] "SQL format" refers to the format of instructions for performing database operations using Structured Query Language (SQL).

[0122] The present invention is a system that improves the efficiency of data processing tasks performed by users through calculation software. This system is configured around a data processing device and provides technology to automate conversion to specific data formats and line break insertion in program code.

[0123] composition

[0124] This system consists of a user terminal, a server, and a generative AI model. The roles and usage of each are explained below.

[0125] User terminal

[0126] A user device is an interface through which users input, receive, and send data. Specific hardware examples include personal computers (PCs), laptops, and tablets. Software examples include calculation software (e.g., Excel) and text editors (e.g., Visual Studio Code).

[0127] server

[0128] The server is the center of data processing and transformation. It analyzes the received data, converts it into SQL format, and inserts line breaks into the program code. The server is a computer system with a high-performance processor and large memory, and dedicated software for running the generative AI model is installed on it.

[0129] Generative AI Models

[0130] A generative AI model is an algorithm that uses machine learning techniques to automate specific data processing tasks. The model resides on a server and performs processes such as converting data formats and inserting line breaks into program code. Specifically, it applies natural language processing and data analysis techniques.

[0131] Data format conversion

[0132] 1. The user uses a calculation software (e.g. Excel) to select and copy the necessary data. For example, they select the columns "Fiscal Year," "Sales," and "Expenses."

[0133] 2. The device retrieves the selected data from the clipboard and prepares it to be sent to the server.

[0134] 3. The server parses the received data and uses a generative AI model to convert the data into a specific SQL format, for example, "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0135] 4. The server returns the converted SQL data to the terminal, which displays it to the user. The user can then check the SQL data and paste it into a database for use.

[0136] Inserting a line break

[0137] 1. A user types SQL code on a terminal or copies and pastes it from another source. For example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000" is entered.

[0138] 2. The device sends the entered code to the server.

[0139] 3. The server parses the received SQL code and uses a generative AI model to insert line breaks in the appropriate places, for example, "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0140] 4. The server returns the SQL code with line breaks to the terminal, which displays it to the user, who can then see the more readable SQL code and use it.

[0141] Specific examples

[0142] Data format conversion example

[0143] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0144] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0145] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0146] Example of inserting a line break

[0147] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0148] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0149] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0150] Prompt Sentence Examples

[0151] "Convert the data selected in Excel into an SQL INSERT statement."

[0152] "Please insert appropriate line breaks into long SQL code."

[0153] As a result of the above, the system of the present invention can improve the efficiency of the user's data processing work and reduce the occurrence of errors.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Processing steps in data format conversion

[0156] Step 1:

[0157] The user opens a calculation software (e.g. Excel), selects the columns of data to be converted (e.g. "Fiscal Year", "Sales", "Expenses") and copies them. This operation saves the selected data to the user's clipboard.

[0158] Step 2:

[0159] The device retrieves data from the user's clipboard. This data is temporarily stored in internal memory, where it is analyzed for format and structure to verify that the data was retrieved correctly.

[0160] Step 3:

[0161] To send the data retrieved from the clipboard to the server, the device packages the data as an HTTP request, typically in JSON format, including the data cue.

[0162] Step 4:

[0163] The server receives the data sent from the device, analyzes the input data, and converts it into the appropriate format for input to the generative AI model. Specifically, it identifies the column names ("Fiscal Year," "Sales," "Expenses") and their corresponding data.

[0164] Step 5:

[0165] The server uses a generative AI model to convert the incoming data into SQL format, for example, generating SQL syntax like "INSERT INTO Sales (Year, Revenue, Expense) VALUES ..." The generative AI model creates the appropriate SQL statement, taking into account column names and data types.

[0166] Step 6:

[0167] The server returns the converted SQL data to the terminal as structured data in JSON format, etc. At this time, the data is sent using a communication protocol (e.g., HTTP).

[0168] Step 7:

[0169] The terminal receives the SQL data returned from the server, interprets it, and displays it to the user. It also formats it as SQL code and displays it in a text editor or browser in a format that is easy for the user to understand.

[0170] Processing steps for inserting line breaks

[0171] Step 1:

[0172] Users enter SQL code into a text editor on their terminal or copy and paste it from another source, for example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0173] Step 2:

[0174] The terminal temporarily stores the entered SQL code in its internal memory, analyzes this code to verify the integrity of the data, and then sends this code to the server.

[0175] Step 3:

[0176] To send the SQL code to the server, the terminal packages the data as an HTTP request, which includes the entire SQL code entered.

[0177] Step 4:

[0178] The server receives the SQL code sent from the device, analyzes the input data, and converts it into an appropriate format for input to the generative AI model. Specifically, it analyzes the grammar and keywords of the SQL statement.

[0179] Step 5:

[0180] The server uses a generative AI model to insert line breaks in the appropriate places, for example, transforming code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;" into "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0181] Step 6:

[0182] The server returns the SQL code after the line break to the terminal as structured data in JSON format or other format, using a communication protocol (e.g. HTTP) to send the data.

[0183] Step 7:

[0184] The terminal receives the SQL code returned from the server, interprets it, and displays it to the user. The SQL code is formatted and displayed in a readable format in a text editor or browser.

[0185] (Application example 1)

[0186] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0187] In modern factory automation, the format conversion of collected data and the management of complex program code are not efficiently carried out. This results in problems such as time-consuming data processing and reduced work efficiency. Furthermore, improving the readability of SQL code requires manual insertion of line breaks, which wastes human resources.

[0188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0189] In this invention, the server includes a means for centrally managing data related to factory automation, a means for automatically converting the data into a required data format, and a means for inserting line breaks into the SQL code to improve readability, thereby making it possible to improve the efficiency of data processing work for factory automation and automate the management of program code.

[0190] A "user terminal" is a device that specifies a data set and provides a receiving and transmitting interface.

[0191] A "dataset" is a user-specified set of data that is to be converted into a particular format.

[0192] A "specific data format" is a format that is compatible with the calculation software or database query used by the user.

[0193] A "generative model" is a machine learning technique used to convert data formats and automatically format program code.

[0194] "Program code" is a series of instructions for executing calculation software or database operations.

[0195] "Inserting a line break" refers to the process of inserting a line break at an appropriate position in the program code to improve readability.

[0196] A "system" is a data management and processing mechanism consisting of a series of data processing devices, user terminals, servers, and generative models.

[0197] "Factory automation" refers to the equipment and devices used to automate tasks within a factory.

[0198] "Centralized management" refers to collecting and managing data in one place, and providing unified access and operation.

[0199] "Processing efficiently" refers to processing data or program code quickly and accurately.

[0200] "SQL code" means statements in a structured query language used to manipulate relational databases.

[0201] To implement this invention, a system consisting of a user terminal, a server, and a generative model is used. The user terminal works in conjunction with the control system of factory robots and industrial equipment, providing an interface for specifying datasets. The server functions as the center of data processing and conversion, converting data into a specific data format using the generative model.

[0202] The user device receives data collected from the factory robots and sends it to a cloud server. This process connects with calculation software and databases that are convenient for managing and visualizing the data. The server analyzes the received dataset and uses a generative model to convert it into a specific data format, such as SQL. The converted dataset is then sent back to the user device, where the user can use it as a database query.

[0203] In a specific implementation, a user first selects the necessary data using the factory robot control system. The selected data is then sent to a server via the cloud, where it is automatically converted into a specific data format using a generative model. For example, factory work data collected in Excel format can be converted into SQL format and returned to the user's device. The server can also automatically insert line breaks in entered SQL code and other program code at appropriate positions to improve readability.

[0204] The required hardware is a user terminal (a factory robot control unit or industrial PC) and a server (a cloud-based data processing server). The software uses the Pandas library, which is useful for data management, and the requests library, which processes HTTP requests. A machine learning model (e.g., GPT-3) is used as the generative model.

[0205] Specific examples are shown below.

[0206] For example, enter the following as the prompt:

[0207] Prompt: Convert the following Excel data into the appropriate SQL format.

[0208] data:

[0209] Year:2023, 2024

[0210] Sales: 10000, 15000

[0211] Cost: 5000, 7000

[0212] Expected output:

[0213] INSERT INTO Sales (Year, Revenue, Expense) VALUES (2023, 10000, 5000), (2024, 15000, 7000);

[0214] This allows for efficient data format conversion and program code processing related to factory automation, thereby improving work efficiency.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] The user selects and specifies the required data using the factory robot control system.

[0218] Input: Work data collected from factory robots.

[0219] Output: A dataset specified by the user's device.

[0220] Specific operation: The user selects and copies data in Excel format, etc.

[0221] Step 2:

[0222] The device retrieves the specified data set from the clipboard and prepares it to be sent to the server.

[0223] Input: The dataset copied to the clipboard.

[0224] Output: The dataset that is sent to the server.

[0225] Specific operation: Retrieves data from the clipboard, converts it to JSON format, and prepares it to be sent to the server.

[0226] Step 3:

[0227] The server receives the dataset sent from the device and converts it into a specific data format using a generative model.

[0228] Input: Dataset submitted in JSON format.

[0229] Output: A dataset converted into a specific data format (e.g., SQL format).

[0230] Specific operation: The server receives the dataset and converts it into SQL format using a generative model (e.g., GPT-3).

[0231] Step 4:

[0232] The server returns the transformed data set to the user terminal.

[0233] Input: A dataset converted into a specific data format.

[0234] Output: The dataset that is sent back to the user's device.

[0235] Specific operation: The converted dataset is sent to the user terminal as a response to the HTTP request.

[0236] Step 5:

[0237] The terminal provides the received converted data to the user and displays it.

[0238] Input: A dataset returned from the server in a specific data format.

[0239] Output: The data format, such as an SQL query, that is made available to the user.

[0240] Specific operation: The converted dataset is displayed on the terminal, and the user pastes it into the database for use.

[0241] Step 6:

[0242] Requires the user to type or copy and paste program code (e.g., SQL code) and insert line breaks at the appropriate places.

[0243] Input: Program code typed or pasted by the user.

[0244] Output: The program code sent from the terminal to the server.

[0245] What it does: Copies the code the user types or pastes to the clipboard and prepares it to be sent to the server.

[0246] Step 7:

[0247] The server analyzes the received program code and uses a generative model to insert line breaks at the appropriate positions.

[0248] Input: The program code sent from the terminal.

[0249] Output: Program code with line breaks inserted.

[0250] Specific operation: The code received by the server is analyzed using a generative model, and line breaks are automatically determined and inserted to improve readability.

[0251] Step 8:

[0252] The server returns the program code with line breaks inserted to the terminal.

[0253] Input: Program code with line breaks inserted.

[0254] Output: Program code returned to the terminal with line breaks.

[0255] Specific operation: The program code after the line break is sent to the terminal as a response to the HTTP request.

[0256] Step 9:

[0257] The terminal displays the program code after the line break to the user.

[0258] Input: The program code returned from the server after the line break.

[0259] Output: Readable program code available to the user.

[0260] Specific operation: The program code after line breaks is displayed on the terminal so that the user can use it as is.

[0261] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0262] This invention relates to a system that not only improves the efficiency of data processing work using computer software, but also provides a feedback adjustment function based on user emotion recognition. Specifically, it is composed of a data processing device that converts data formats, inserts line breaks into program code, and recognizes emotions using an emotion engine.

[0263] composition

[0264] This system consists of a user device, a server, a generative model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[0265] What the program does

[0266] Data format conversion

[0267] 1. Data selection and acquisition

[0268] The user selects the required data in a calculation software (e.g. Excel). After selection, the data is copied.

[0269] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[0270] 2. Data transmission and conversion

[0271] The device sends the acquired data to the server via an HTTP request. The data is packaged in JSON format or similar.

[0272] The server parses the received data and converts it into a specific SQL format using a generative model.

[0273] 3. Return and display of converted data

[0274] The server sends the converted SQL data to the terminal as an HTTP response.

[0275] The terminal displays the received SQL data to the user.

[0276] Inserting a line break

[0277] 1. Enter and submit SQL code

[0278] The user types or pastes SQL code into the terminal.

[0279] The terminal sends the entered code to the server.

[0280] 2. Line break handling

[0281] The server parses the received SQL code and uses the generative model to insert line breaks at the appropriate places.

[0282] 3. Returning and displaying the code after the line break

[0283] The server sends the SQL code with line breaks inserted to the terminal.

[0284] The terminal re-displays the received SQL code after line breaks in the input field.

[0285] Emotion recognition by emotion engine

[0286] 1. Collecting Emotional Data

[0287] The device uses a microphone and camera to collect the user's voice and facial expressions.

[0288] 2. Emotion Recognition Analysis

[0289] The server uses an emotion engine to recognize the user's emotional state from the collected voice and facial expression data, for example classifying emotions as positive, negative, neutral, etc.

[0290] 3. Adjusting feedback

[0291] The server adjusts the feedback of the data processing based on the recognized emotion, for example, providing more detailed explanations or guidance if the user is showing negative emotions.

[0292] 4. Providing Feedback

[0293] The terminal provides the user with the feedback received from the server, thereby improving the user's work efficiency and satisfaction.

[0294] Specific examples

[0295] Data format conversion example

[0296] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0297] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0298] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0299] Example of inserting a line break

[0300] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0301] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0302] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0303] Example of feedback adjustment using emotion recognition

[0304] While the user is using the system, the terminal collects the user's voice and facial expressions.

[0305] The server uses an emotion engine to recognize that the user is confused.

[0306] The server generates a more detailed help message and sends it to the terminal.

[0307] The terminal displays a help message to the user, and the user receives assistance in resolving the problem.

[0308] The configuration and processing of the present invention allow users to quickly and efficiently convert data into specific data formats and improve the readability of program code, and also provide a better user experience through feedback adjustment based on emotion recognition.

[0309] The processing flow will be explained below.

[0310] Data format conversion

[0311] Step 1:

[0312] The user selects the required data in calculation software (e.g. Excel). For example, the user selects a range of columns such as "Fiscal Year," "Sales," and "Expenses."

[0313] Step 2:

[0314] Copies the data selected by the user. The selected data is transferred to the clipboard.

[0315] Step 3:

[0316] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[0317] Step 4:

[0318] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[0319] Step 5:

[0320] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[0321] Step 6:

[0322] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0323] Step 7:

[0324] The server sends the converted SQL data to the terminal as an HTTP response.

[0325] Step 8:

[0326] The terminal displays the received SQL data to the user in a dedicated window or message box.

[0327] Step 9:

[0328] The user checks the displayed SQL data and pastes it into a database for use.

[0329] Inserting a line break

[0330] Step 1:

[0331] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0332] Step 2:

[0333] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[0334] Step 3:

[0335] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[0336] Step 4:

[0337] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[0338] Step 5:

[0339] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[0340] Step 6:

[0341] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[0342] Step 7:

[0343] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[0344] Step 8:

[0345] The user can review the displayed SQL code and edit or use it as needed.

[0346] Emotion recognition and feedback adjustment by emotion engine

[0347] Step 1:

[0348] The device collects the user's voice and facial expression data in real time using a microphone and camera, for example, capturing the user's face with a webcam.

[0349] Step 2:

[0350] The device will transmit the collected voice and facial expression data to a server, using appropriate data security techniques.

[0351] Step 3:

[0352] The server uses an emotion engine to analyze the user's voice and facial expression data, for example, by evaluating voice tone and facial muscle movements to identify emotions.

[0353] Step 4:

[0354] The server adjusts the feedback of the data processing based on the perceived emotion, for example generating a more detailed help message if it determines that the user is confused.

[0355] Step 5:

[0356] Sends server-generated feedback to the device, which may include additional explanations or guidance.

[0357] Step 6:

[0358] The device displays the received feedback to the user, allowing the user to work more efficiently.

[0359] The above are the specific processing steps for data format conversion, line break insertion, and feedback adjustment through emotion recognition in this system.

[0360] Example 2

[0361] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0362] In conventional data processing systems, the format conversion of user-specified data sets and the formatting of program code are often performed manually, which reduces efficiency and leads to errors.In addition, one-sided feedback is provided without considering the user's emotional state, which leads to a lack of user satisfaction.

[0363] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0364] In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative AI model to convert the dataset into a specific data format, means for acquiring data from a clipboard, means for providing the converted dataset to the user, means for collecting voice and facial expressions to recognize the emotional state of the user, and means for recognizing emotions from the collected data using an emotion engine and adjusting feedback based on the recognized emotions. This allows for efficient automatic data conversion and program code formatting, and further enables appropriate feedback to be provided based on the user's emotional state.

[0365] A "dataset" is a set of data that a user specifies they want to process.

[0366] A "generative AI model" is an algorithm that uses machine learning techniques to generate specific outputs based on large amounts of data.

[0367] The "clipboard" is a memory area that temporarily stores data within a terminal and enables data transfer between other applications.

[0368] "Emotion engine" is a general term for algorithms and software that analyze voice and facial expression data to identify a user's emotional state.

[0369] "Feedback" refers to the reactions and information provided by a system in response to a user's actions and emotions.

[0370] A "user" is a person or entity that is responsible for operating the system and providing data and instructions.

[0371] The present invention is a system that utilizes computer software to provide efficient data processing and feedback adjustment based on user emotion recognition. Specifically, it constitutes a data processing device that includes data format conversion, line break insertion in program code, and emotion recognition using an emotion engine.

[0372] composition

[0373] This system consists of a user device, a server, a generative AI model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative AI model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[0374] Data format conversion

[0375] In data format conversion, the user selects the required data in calculation software (e.g., Excel) and copies it to the clipboard. The terminal retrieves the data from the clipboard and sends it to the server. The server analyzes the received data and converts it into SQL format (e.g., INSERT INTO Sales (Year, Revenue, Expense) VALUES ...) using a generative AI model. The converted SQL data is sent back from the server to the terminal and displayed to the user. An example of an extracted prompt sentence is as follows:

[0376] Example prompt sentence:

[0377] Convert data containing columns such as "Year", "Revenue", and "Expense" into SQL format (e.g. INSERT INTO Sales (Year, Revenue, Expense) VALUES ...).

[0378] Inserting a line break

[0379] To insert a line break into SQL code, the user enters or pastes the SQL code on the terminal. The terminal sends the entered code to the server, which analyzes the received SQL code and uses a generative AI model to insert line breaks in the appropriate positions. The SQL code with line breaks is then sent back to the terminal and redisplayed to the user. An example of an extracted prompt sentence is as follows:

[0380] Example prompt sentence:

[0381] Insert line breaks at the appropriate positions in the following SQL code: Code: SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;

[0382] Emotion recognition by emotion engine

[0383] In emotion recognition using an emotion engine, the device uses a microphone and camera to collect the user's voice and facial expression data. The collected data is sent to the server in real time, where it is analyzed and used by the emotion engine to recognize the user's emotional state. For example, emotions are classified as positive, negative, or neutral. Based on the recognized emotional state, the server adjusts the feedback and generates detailed guidance and explanations. The generated feedback is sent to the device and displayed to the user. An example of an extracted prompt sentence is as follows:

[0384] Example prompt sentence:

[0385] Analyze voice and facial expression data to classify the user's emotional state (positive, negative, neutral, etc.). If it is negative, generate a detailed help message to provide feedback.

[0386] This allows users to quickly and efficiently convert data into specific formats, improve the readability of program code, and receive appropriate feedback through emotion recognition.

[0387] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0388] Data format conversion

[0389] Step 1:

[0390] The user opens a calculation software (e.g. Excel) and selects the columns "Year", "Sales", and "Expenses" to copy. The input of this step is the data selected by the user, and the output is the data copied to the clipboard.

[0391] Step 2:

[0392] The terminal gets data from the clipboard and sends it to the server. The input of this step is the data stored in the clipboard, and the output is the JSON formatted data that is sent to the server.

[0393] Step 3:

[0394] The server parses the received data and converts it into a specific SQL format using a generative AI model. The input of this step is JSON formatted data and the output is SQL formatted data.

[0395] Step 4:

[0396] The server sends the converted SQL data to the terminal as an HTTP response. The input of this step is data in SQL format, and the output is the data sent to the terminal as an HTTP response.

[0397] Step 5:

[0398] The terminal formats the received SQL data and displays it to the user. The input to this step is the SQL data received from the server, and the output is the formatted SQL data that is displayed to the user.

[0399] Inserting a line break

[0400] Step 1:

[0401] The user types SQL code into their terminal (e.g., "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000") or pastes it from the clipboard. The input for this step is the SQL code typed or pasted by the user, and the output is the SQL code displayed on the terminal.

[0402] Step 2:

[0403] The terminal sends the entered or pasted SQL code to the server. The input of this step is the SQL code entered or pasted by the user, and the output is the SQL code sent to the server.

[0404] Step 3:

[0405] The server parses the received SQL code and uses a generative AI model to insert line breaks at the appropriate positions. The input to this step is the received SQL code, and the output is the SQL code with line breaks inserted.

[0406] Step 4:

[0407] The server sends the SQL code with line breaks inserted to the terminal. The input of this step is the SQL code with line breaks inserted, and the output is the SQL code with line breaks sent to the terminal.

[0408] Step 5:

[0409] The terminal re-displays the received SQL code with line breaks to the user. The input of this step is the SQL code received from the server with line breaks, and the output is the SQL code with line breaks re-displayed to the user.

[0410] Emotion recognition by emotion engine

[0411] Step 1:

[0412] The device uses a microphone and a camera to collect the user's voice and facial expressions. The input of this step is the user's voice and facial expression data, and the output is the collected real-time emotion data.

[0413] Step 2:

[0414] The terminal transmits the collected emotion data to the server in real time. The input of this step is the collected emotion data, and the output is the emotion data transmitted to the server.

[0415] Step 3:

[0416] The server uses an emotion engine to recognize the user's emotional state from the transmitted voice and facial expression data. The input of this step is the collected voice and facial expression data, and the output is the recognized user's emotional state (e.g., positive, negative, neutral).

[0417] Step 4:

[0418] The server adjusts the feedback based on the recognized emotional state and generates detailed guides and explanations. The input of this step is the recognized emotional state and the output is the adjusted feedback data.

[0419] Step 5:

[0420] The server sends the generated feedback data to the terminal. The input of this step is the generated feedback data, and the output is the feedback data sent to the terminal.

[0421] Step 6:

[0422] The terminal displays the feedback received from the server to the user. The input of this step is the feedback data received from the server, and the output is the feedback displayed to the user.

[0423] (Application example 2)

[0424] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0425] Conventional data processing systems have limitations in converting user-specified data sets and improving the readability of program code. It is also difficult for systems to recognize users' emotional states and adjust feedback in real time, which is particularly important for improving the quality of customer service in brick-and-mortar stores. The present invention aims to solve these problems and significantly improve the user experience.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative model to convert the dataset into a specific data format, means for providing the converted dataset to the user, means for using an emotion engine to recognize the emotional state of the user, and means for adjusting feedback based on the recognized emotional state. This enables the system to quickly and efficiently convert into a data format, improve the readability of program code, and adjust feedback based on emotion recognition.

[0427] A "dataset" refers to a series of data items or information specified by a user.

[0428] A "generative model" is a model that uses machine learning algorithms to transform or reorganize data into a specific format.

[0429] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state from input data such as voice and facial expressions and recognizes that state.

[0430] "Feedback" refers to the responses or instructions provided by a system based on a user's input or emotional state.

[0431] "Server" refers to a computer system that acts as a data processing and transformation center.

[0432] The system for implementing this invention consists of four main components: a server, a user terminal, a generative model, and an emotion engine. Below, we will provide an overview of the functions and processing of each of these components.

[0433] System Configuration

[0434] The system has the ability to receive user-specified data sets and convert them into specific data formats, as well as to insert line breaks at appropriate positions in user-entered program code and to recognize the user's emotional state and adjust feedback accordingly.

[0435] User terminal

[0436] The user terminal can use smart glasses and function as an interface for data input, reception, and transmission. This allows the user's voice and facial expressions to be collected in real time and transmitted to the server. The server also displays feedback to the user.

[0437] server

[0438] The server acts as a data processing and transformation center, using a generative model to transform the data and returning the converted dataset to the user in a specific data format. It also uses an emotion engine to recognize the user's emotional state and generate feedback based on that.

[0439] Generative Model

[0440] Generative models are used to transform data into specific formats or to insert line breaks into program code. They use machine learning algorithms to transform input data efficiently and accurately.

[0441] Emotion Engine

[0442] The emotion engine analyzes and recognizes the user's emotional state from input data such as voice and facial expressions, and if the emotional state is negative, it provides the user with detailed guidance and support messages.

[0443] System Processing Overview

[0444] Data format conversion

[0445] 1. Receiving a user-specified dataset

[0446] The user inputs a dataset using voice commands through the smart glasses. The device receives the data and prepares it for transmission to the server.

[0447] 2. Data format conversion by the server

[0448] The server uses the generative model to transform the received dataset into a specific format, for example, into an SQL query or a specific database format.

[0449] Emotion Recognition and Feedback

[0450] 1. Recognizing the user's emotional state

[0451] The smart glasses use a built-in camera and microphone to collect the user's voice and facial expressions, and the server uses an emotion engine to recognize the user's emotional state from these data.

[0452] 2. Adjusting feedback

[0453] The server adjusts the feedback based on the perceived emotional state, providing detailed help messages and direct support if negative emotions are detected.

[0454] Specific examples

[0455] For example, when a user voice-inputs a customer's order details through the smart glasses, the data set is sent from the smart glasses to the server and converted into an SQL query format. If the user's facial expression is confused, the server's emotion engine will recognize it and display a detailed guide message on the smart glasses.

[0456] Prompt Sentence Examples

[0457] Get customer order details and convert them into SQL format. Also, recognize the customer's emotional state and provide a support message if necessary.

[0458] This system is expected to improve customer service efficiency in physical stores and increase customer satisfaction.

[0459] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0460] Step 1:

[0461] The user specifies the customer's order by voice input via the smart glasses. The microphone in the smart glasses captures the voice data and converts it into text data using voice recognition software (e.g., Google Cloud Speech-to-Text). This text data is then input into the user's device.

[0462] Step 2:

[0463] The user device packages the acquired text data in JSON format and sends it to the server using an HTTP request. The server receives the request and parses the data using a JSON parser (e.g., json.loads).

[0464] Step 3:

[0465] The server inputs the analyzed data into a generative model (e.g., GPT-3) and performs data processing to convert it into a specific data format (e.g., SQL query). The server receives the SQL query output by the generative model.

[0466] Step 4:

[0467] The server sends the generated SQL query as an HTTP response to the user terminal, which receives the HTTP response and displays the acquired SQL query on the display of the smart glasses.

[0468] Step 5:

[0469] The smart glasses' camera and microphone are used to collect the customer's voice and facial expressions in real time, and this data is then input into the device, which then sends it to the server as an HTTP request.

[0470] Step 6:

[0471] The server inputs the received voice and facial expression data into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze and recognize the customer's emotional state. The emotion engine outputs the emotion recognition results to the server.

[0472] Step 7:

[0473] The server generates feedback based on the recognized emotional state. For example, if the customer is confused, it generates a detailed guide message. The server sends this feedback to the terminal as an HTTP response.

[0474] Step 8:

[0475] The user terminal displays the feedback received from the server on the display of the smart glasses, for example, a message such as "What's the matter? We'll show you around."

[0476] This series of processes is expected to significantly improve the quality and efficiency of customer service.

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] The present invention provides a system that improves the efficiency of data processing tasks performed by users through calculation software. Specifically, it provides technology that automates data format conversion and line break insertion in program code, focusing on data processing devices.

[0494] composition

[0495] This system consists of a user terminal, a server, and a generative model. The user terminal serves as the interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques.

[0496] What the program does

[0497] Data format conversion

[0498] 1. Data selection and acquisition

[0499] The user selects the required data using calculation software (e.g. Excel), and the data is then copied.

[0500] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[0501] 2. Data transmission and conversion

[0502] The terminal transmits the acquired data to the server.

[0503] The server parses the received data and converts it into a specific SQL format using a generative model, which includes extracting column names and generating the appropriate SQL syntax.

[0504] 3. Return and display of converted data

[0505] The server returns the converted SQL data to the terminal.

[0506] The terminal displays the received SQL data to the user, who can then check it and use it as an SQL query.

[0507] Inserting a line break

[0508] 1. Enter and submit SQL code

[0509] The user types or copies and pastes the SQL code into the terminal.

[0510] The terminal sends the entered code to the server.

[0511] 2. Line break handling

[0512] The server parses the received SQL code and uses a generative model to insert line breaks at appropriate places, a process that automatically determines where to break lines to improve readability.

[0513] 3. Return and display of code after line break

[0514] The server returns the SQL code with line breaks inserted to the terminal.

[0515] The terminal displays the received SQL code with line breaks to the user, allowing the user to check the SQL code with improved readability.

[0516] Specific examples

[0517] Data format conversion example

[0518] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0519] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0520] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0521] Example of inserting a line break

[0522] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0523] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0524] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0525] The configuration and processing of the present invention allow the user to quickly and efficiently convert data into a specific data format and improve the readability of program code.

[0526] The processing flow will be explained below.

[0527] Data format conversion

[0528] Step 1:

[0529] The user selects the necessary data in calculation software (e.g., Excel). For example, the user selects the range of columns "Fiscal Year," "Sales," and "Expenses" in an Excel sheet.

[0530] Step 2:

[0531] Copies the data selected by the user. The selected data is transferred to the clipboard.

[0532] Step 3:

[0533] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[0534] Step 4:

[0535] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[0536] Step 5:

[0537] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[0538] Step 6:

[0539] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0540] Step 7:

[0541] The server sends the converted SQL data to the terminal as an HTTP response.

[0542] Step 8:

[0543] The terminal displays the received SQL data to the user in a dedicated window or message box.

[0544] Step 9:

[0545] The user checks the displayed SQL data and pastes it into a database for use.

[0546] Inserting a line break

[0547] Step 1:

[0548] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0549] Step 2:

[0550] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[0551] Step 3:

[0552] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[0553] Step 4:

[0554] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[0555] Step 5:

[0556] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[0557] Step 6:

[0558] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[0559] Step 7:

[0560] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[0561] Step 8:

[0562] The user can review the displayed SQL code and edit or use it as needed.

[0563] The above are the specific processing steps for data format conversion and line break insertion in this system.

[0564] Example 1

[0565] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0566] Conventional data processing devices require users to manually convert data into a specific format, which requires time and effort. Furthermore, manually inserting line breaks to improve program code readability is cumbersome and inefficient. This reduces user work efficiency and increases the likelihood of errors.

[0567] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0568] In this invention, the server includes a means for receiving a dataset specified by a user, a means for using a generative AI model to convert the dataset into a specific data format, and a means for providing the converted dataset to the user. This allows the user to convert data formats quickly and accurately. The server also includes a means for using a generative AI model to insert line breaks at appropriate positions in input program code, and a means for returning the code after the line breaks to the user terminal. This improves the readability of the program code and further enhances the user's work efficiency.

[0569] "User" means a person or entity that uses the system to process data or manipulate program code.

[0570] A "dataset" is a set of data that is subject to processing or transformation and may contain data items in different formats.

[0571] A "generative AI model" is a collection of algorithms that use machine learning techniques to automate specific tasks (such as converting data formats or inserting line breaks into code).

[0572] "Receiving means" refers to means for transmitting a data set or program code designated by a user to the system, and includes network communication and the like.

[0573] A "transformation means" is a means that performs a procedure or process to convert a received dataset into a particular format, typically using a generative AI model.

[0574] The "provision means" is a means for presenting the converted results to the user, and is carried out through a display device or a communication device.

[0575] The "clipboard" is a temporary data storage area that the user uses for copy and paste operations.

[0576] A "server" is a computer system that receives, analyzes, transforms, and returns data and has the processing power to run a generative AI model.

[0577] A "terminal" is a computer device that is directly operated by a user, and is a device for inputting data and displaying results.

[0578] "SQL format" refers to the format of instructions for performing database operations using Structured Query Language (SQL).

[0579] The present invention is a system that improves the efficiency of data processing tasks performed by users through calculation software. This system is configured around a data processing device and provides technology to automate conversion to specific data formats and line break insertion in program code.

[0580] composition

[0581] This system consists of a user terminal, a server, and a generative AI model. The roles and usage of each are explained below.

[0582] User terminal

[0583] A user device is an interface through which users input, receive, and send data. Specific hardware examples include personal computers (PCs), laptops, and tablets. Software examples include calculation software (e.g., Excel) and text editors (e.g., Visual Studio Code).

[0584] server

[0585] The server is the center of data processing and transformation. It analyzes the received data, converts it into SQL format, and inserts line breaks into the program code. The server is a computer system with a high-performance processor and large memory, and dedicated software for running the generative AI model is installed on it.

[0586] Generative AI Models

[0587] A generative AI model is an algorithm that uses machine learning techniques to automate specific data processing tasks. The model resides on a server and performs processes such as converting data formats and inserting line breaks into program code. Specifically, it applies natural language processing and data analysis techniques.

[0588] Data format conversion

[0589] 1. The user uses a calculation software (e.g. Excel) to select and copy the necessary data. For example, they select the columns "Fiscal Year," "Sales," and "Expenses."

[0590] 2. The device retrieves the selected data from the clipboard and prepares it to be sent to the server.

[0591] 3. The server parses the received data and uses a generative AI model to convert the data into a specific SQL format, for example, "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0592] 4. The server returns the converted SQL data to the terminal, which displays it to the user. The user can then check the SQL data and paste it into a database for use.

[0593] Inserting a line break

[0594] 1. A user types SQL code on a terminal or copies and pastes it from another source. For example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000" is entered.

[0595] 2. The device sends the entered code to the server.

[0596] 3. The server parses the received SQL code and uses a generative AI model to insert line breaks in the appropriate places, for example, "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0597] 4. The server returns the SQL code with line breaks to the terminal, which displays it to the user, who can then see the more readable SQL code and use it.

[0598] Specific examples

[0599] Data format conversion example

[0600] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0601] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0602] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0603] Example of inserting a line break

[0604] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0605] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0606] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0607] Prompt Sentence Examples

[0608] "Convert the data selected in Excel into an SQL INSERT statement."

[0609] "Please insert appropriate line breaks into long SQL code."

[0610] As a result of the above, the system of the present invention can improve the efficiency of the user's data processing work and reduce the occurrence of errors.

[0611] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0612] Processing steps in data format conversion

[0613] Step 1:

[0614] The user opens a calculation software (e.g. Excel), selects the columns of data to be converted (e.g. "Fiscal Year", "Sales", "Expenses") and copies them. This operation saves the selected data to the user's clipboard.

[0615] Step 2:

[0616] The device retrieves data from the user's clipboard. This data is temporarily stored in internal memory, where it is analyzed for format and structure to verify that the data was retrieved correctly.

[0617] Step 3:

[0618] To send the data retrieved from the clipboard to the server, the device packages the data as an HTTP request, typically in JSON format, including the data cue.

[0619] Step 4:

[0620] The server receives the data sent from the device, analyzes the input data, and converts it into the appropriate format for input to the generative AI model. Specifically, it identifies the column names ("Fiscal Year," "Sales," "Expenses") and their corresponding data.

[0621] Step 5:

[0622] The server uses a generative AI model to convert the incoming data into SQL format, for example, generating SQL syntax like "INSERT INTO Sales (Year, Revenue, Expense) VALUES ..." The generative AI model creates the appropriate SQL statement, taking into account column names and data types.

[0623] Step 6:

[0624] The server returns the converted SQL data to the terminal as structured data in JSON format, etc. At this time, the data is sent using a communication protocol (e.g., HTTP).

[0625] Step 7:

[0626] The terminal receives the SQL data returned from the server, interprets it, and displays it to the user. It also formats it as SQL code and displays it in a text editor or browser in a format that is easy for the user to understand.

[0627] Processing steps for inserting line breaks

[0628] Step 1:

[0629] Users enter SQL code into a text editor on their terminal or copy and paste it from another source, for example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0630] Step 2:

[0631] The terminal temporarily stores the entered SQL code in its internal memory, analyzes this code to verify the integrity of the data, and then sends this code to the server.

[0632] Step 3:

[0633] To send the SQL code to the server, the terminal packages the data as an HTTP request, which includes the entire SQL code entered.

[0634] Step 4:

[0635] The server receives the SQL code sent from the device, analyzes the input data, and converts it into an appropriate format for input to the generative AI model. Specifically, it analyzes the grammar and keywords of the SQL statement.

[0636] Step 5:

[0637] The server uses a generative AI model to insert line breaks in the appropriate places, for example, transforming code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;" into "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0638] Step 6:

[0639] The server returns the SQL code after the line break to the terminal as structured data in JSON format or other format, using a communication protocol (e.g. HTTP) to send the data.

[0640] Step 7:

[0641] The terminal receives the SQL code returned from the server, interprets it, and displays it to the user. The SQL code is formatted and displayed in a readable format in a text editor or browser.

[0642] (Application example 1)

[0643] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0644] In modern factory automation, the format conversion of collected data and the management of complex program code are not efficiently carried out. This results in problems such as time-consuming data processing and reduced work efficiency. Furthermore, improving the readability of SQL code requires manual insertion of line breaks, which wastes human resources.

[0645] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0646] In this invention, the server includes a means for centrally managing data related to factory automation, a means for automatically converting the data into a required data format, and a means for inserting line breaks into the SQL code to improve readability, thereby making it possible to improve the efficiency of data processing work for factory automation and automate the management of program code.

[0647] A "user terminal" is a device that specifies a data set and provides a receiving and transmitting interface.

[0648] A "dataset" is a user-specified set of data that is to be converted into a particular format.

[0649] A "specific data format" is a format that is compatible with the calculation software or database query used by the user.

[0650] A "generative model" is a machine learning technique used to convert data formats and automatically format program code.

[0651] "Program code" is a series of instructions for executing calculation software or database operations.

[0652] "Inserting a line break" refers to the process of inserting a line break at an appropriate position in the program code to improve readability.

[0653] A "system" is a data management and processing mechanism consisting of a series of data processing devices, user terminals, servers, and generative models.

[0654] "Factory automation" refers to the equipment and devices used to automate tasks within a factory.

[0655] "Centralized management" refers to collecting and managing data in one place, and providing unified access and operation.

[0656] "Processing efficiently" refers to processing data or program code quickly and accurately.

[0657] "SQL code" means statements in a structured query language used to manipulate relational databases.

[0658] To implement this invention, a system consisting of a user terminal, a server, and a generative model is used. The user terminal works in conjunction with the control system of factory robots and industrial equipment, providing an interface for specifying datasets. The server functions as the center of data processing and conversion, converting data into a specific data format using the generative model.

[0659] The user device receives data collected from the factory robots and sends it to a cloud server. This process connects with calculation software and databases that are convenient for managing and visualizing the data. The server analyzes the received dataset and uses a generative model to convert it into a specific data format, such as SQL. The converted dataset is then sent back to the user device, where the user can use it as a database query.

[0660] In a specific implementation, a user first selects the necessary data using the factory robot control system. The selected data is then sent to a server via the cloud, where it is automatically converted into a specific data format using a generative model. For example, factory work data collected in Excel format can be converted into SQL format and returned to the user's device. The server can also automatically insert line breaks in entered SQL code and other program code at appropriate positions to improve readability.

[0661] The required hardware is a user terminal (a factory robot control unit or industrial PC) and a server (a cloud-based data processing server). The software uses the Pandas library, which is useful for data management, and the requests library, which processes HTTP requests. A machine learning model (e.g., GPT-3) is used as the generative model.

[0662] Specific examples are shown below.

[0663] For example, enter the following as the prompt:

[0664] Prompt: Convert the following Excel data into the appropriate SQL format.

[0665] data:

[0666] Year:2023, 2024

[0667] Sales: 10000, 15000

[0668] Cost: 5000, 7000

[0669] Expected output:

[0670] INSERT INTO Sales (Year, Revenue, Expense) VALUES (2023, 10000, 5000), (2024, 15000, 7000);

[0671] This allows for efficient data format conversion and program code processing related to factory automation, thereby improving work efficiency.

[0672] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0673] Step 1:

[0674] The user selects and specifies the required data using the factory robot control system.

[0675] Input: Work data collected from factory robots.

[0676] Output: A dataset specified by the user's device.

[0677] Specific operation: The user selects and copies data in Excel format, etc.

[0678] Step 2:

[0679] The device retrieves the specified data set from the clipboard and prepares it to be sent to the server.

[0680] Input: The dataset copied to the clipboard.

[0681] Output: The dataset that is sent to the server.

[0682] Specific operation: Retrieves data from the clipboard, converts it to JSON format, and prepares it to be sent to the server.

[0683] Step 3:

[0684] The server receives the dataset sent from the device and converts it into a specific data format using a generative model.

[0685] Input: Dataset submitted in JSON format.

[0686] Output: A dataset converted into a specific data format (e.g., SQL format).

[0687] Specific operation: The server receives the dataset and converts it into SQL format using a generative model (e.g., GPT-3).

[0688] Step 4:

[0689] The server returns the transformed data set to the user terminal.

[0690] Input: A dataset converted into a specific data format.

[0691] Output: The dataset that is sent back to the user's device.

[0692] Specific operation: The converted dataset is sent to the user terminal as a response to the HTTP request.

[0693] Step 5:

[0694] The terminal provides the received converted data to the user and displays it.

[0695] Input: A dataset returned from the server in a specific data format.

[0696] Output: The data format, such as an SQL query, that is made available to the user.

[0697] Specific operation: The converted dataset is displayed on the terminal, and the user pastes it into the database for use.

[0698] Step 6:

[0699] Requires the user to type or copy and paste program code (e.g., SQL code) and insert line breaks at the appropriate places.

[0700] Input: Program code typed or pasted by the user.

[0701] Output: The program code sent from the terminal to the server.

[0702] What it does: Copies the code the user types or pastes to the clipboard and prepares it to be sent to the server.

[0703] Step 7:

[0704] The server analyzes the received program code and uses a generative model to insert line breaks at the appropriate positions.

[0705] Input: The program code sent from the terminal.

[0706] Output: Program code with line breaks inserted.

[0707] Specific operation: The code received by the server is analyzed using a generative model, and line breaks are automatically determined and inserted to improve readability.

[0708] Step 8:

[0709] The server returns the program code with line breaks inserted to the terminal.

[0710] Input: Program code with line breaks inserted.

[0711] Output: Program code returned to the terminal with line breaks.

[0712] Specific operation: The program code after the line break is sent to the terminal as a response to the HTTP request.

[0713] Step 9:

[0714] The terminal displays the program code after the line break to the user.

[0715] Input: The program code returned from the server after the line break.

[0716] Output: Readable program code available to the user.

[0717] Specific operation: The program code after line breaks is displayed on the terminal so that the user can use it as is.

[0718] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0719] This invention relates to a system that not only improves the efficiency of data processing work using computer software, but also provides a feedback adjustment function based on user emotion recognition. Specifically, it is composed of a data processing device that converts data formats, inserts line breaks into program code, and recognizes emotions using an emotion engine.

[0720] composition

[0721] This system consists of a user device, a server, a generative model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[0722] What the program does

[0723] Data format conversion

[0724] 1. Data selection and acquisition

[0725] The user selects the required data in a calculation software (e.g. Excel). After selection, the data is copied.

[0726] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[0727] 2. Data transmission and conversion

[0728] The device sends the acquired data to the server via an HTTP request. The data is packaged in JSON format or similar.

[0729] The server parses the received data and converts it into a specific SQL format using a generative model.

[0730] 3. Return and display of converted data

[0731] The server sends the converted SQL data to the terminal as an HTTP response.

[0732] The terminal displays the received SQL data to the user.

[0733] Inserting a line break

[0734] 1. Enter and submit SQL code

[0735] The user types or pastes SQL code into the terminal.

[0736] The terminal sends the entered code to the server.

[0737] 2. Line break handling

[0738] The server parses the received SQL code and uses the generative model to insert line breaks at the appropriate places.

[0739] 3. Returning and displaying the code after the line break

[0740] The server sends the SQL code with line breaks inserted to the terminal.

[0741] The terminal re-displays the received SQL code after line breaks in the input field.

[0742] Emotion recognition by emotion engine

[0743] 1. Collecting Emotional Data

[0744] The device uses a microphone and camera to collect the user's voice and facial expressions.

[0745] 2. Emotion Recognition Analysis

[0746] The server uses an emotion engine to recognize the user's emotional state from the collected voice and facial expression data, for example classifying emotions as positive, negative, neutral, etc.

[0747] 3. Adjusting feedback

[0748] The server adjusts the feedback of the data processing based on the recognized emotion, for example, providing more detailed explanations or guidance if the user is showing negative emotions.

[0749] 4. Providing Feedback

[0750] The terminal provides the user with the feedback received from the server, thereby improving the user's work efficiency and satisfaction.

[0751] Specific examples

[0752] Data format conversion example

[0753] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0754] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0755] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0756] Example of inserting a line break

[0757] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0758] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0759] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0760] Example of feedback adjustment using emotion recognition

[0761] While the user is using the system, the terminal collects the user's voice and facial expressions.

[0762] The server uses an emotion engine to recognize that the user is confused.

[0763] The server generates a more detailed help message and sends it to the terminal.

[0764] The terminal displays a help message to the user, and the user receives assistance in resolving the problem.

[0765] The configuration and processing of the present invention allow users to quickly and efficiently convert data into specific data formats and improve the readability of program code, and also provide a better user experience through feedback adjustment based on emotion recognition.

[0766] The processing flow will be explained below.

[0767] Data format conversion

[0768] Step 1:

[0769] The user selects the required data in calculation software (e.g. Excel). For example, the user selects a range of columns such as "Fiscal Year," "Sales," and "Expenses."

[0770] Step 2:

[0771] Copies the data selected by the user. The selected data is transferred to the clipboard.

[0772] Step 3:

[0773] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[0774] Step 4:

[0775] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[0776] Step 5:

[0777] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[0778] Step 6:

[0779] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0780] Step 7:

[0781] The server sends the converted SQL data to the terminal as an HTTP response.

[0782] Step 8:

[0783] The terminal displays the received SQL data to the user in a dedicated window or message box.

[0784] Step 9:

[0785] The user checks the displayed SQL data and pastes it into a database for use.

[0786] Inserting a line break

[0787] Step 1:

[0788] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0789] Step 2:

[0790] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[0791] Step 3:

[0792] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[0793] Step 4:

[0794] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[0795] Step 5:

[0796] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[0797] Step 6:

[0798] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[0799] Step 7:

[0800] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[0801] Step 8:

[0802] The user can review the displayed SQL code and edit or use it as needed.

[0803] Emotion recognition and feedback adjustment by emotion engine

[0804] Step 1:

[0805] The device collects the user's voice and facial expression data in real time using a microphone and camera, for example, capturing the user's face with a webcam.

[0806] Step 2:

[0807] The device will transmit the collected voice and facial expression data to a server, using appropriate data security techniques.

[0808] Step 3:

[0809] The server uses an emotion engine to analyze the user's voice and facial expression data, for example, by evaluating voice tone and facial muscle movements to identify emotions.

[0810] Step 4:

[0811] The server adjusts the feedback of the data processing based on the perceived emotion, for example generating a more detailed help message if it determines that the user is confused.

[0812] Step 5:

[0813] Sends server-generated feedback to the device, which may include additional explanations or guidance.

[0814] Step 6:

[0815] The device displays the received feedback to the user, allowing the user to work more efficiently.

[0816] The above are the specific processing steps for data format conversion, line break insertion, and feedback adjustment through emotion recognition in this system.

[0817] Example 2

[0818] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0819] In conventional data processing systems, the format conversion of user-specified data sets and the formatting of program code are often performed manually, which reduces efficiency and leads to errors.In addition, one-sided feedback is provided without considering the user's emotional state, which leads to a lack of user satisfaction.

[0820] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0821] In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative AI model to convert the dataset into a specific data format, means for acquiring data from a clipboard, means for providing the converted dataset to the user, means for collecting voice and facial expressions to recognize the emotional state of the user, and means for recognizing emotions from the collected data using an emotion engine and adjusting feedback based on the recognized emotions. This allows for efficient automatic data conversion and program code formatting, and further enables appropriate feedback to be provided based on the user's emotional state.

[0822] A "dataset" is a set of data that a user specifies they want to process.

[0823] A "generative AI model" is an algorithm that uses machine learning techniques to generate specific outputs based on large amounts of data.

[0824] The "clipboard" is a memory area that temporarily stores data within a terminal and enables data transfer between other applications.

[0825] "Emotion engine" is a general term for algorithms and software that analyze voice and facial expression data to identify a user's emotional state.

[0826] "Feedback" refers to the reactions and information provided by a system in response to a user's actions and emotions.

[0827] A "user" is a person or entity that is responsible for operating the system and providing data and instructions.

[0828] The present invention is a system that utilizes computer software to provide efficient data processing and feedback adjustment based on user emotion recognition. Specifically, it constitutes a data processing device that includes data format conversion, line break insertion in program code, and emotion recognition using an emotion engine.

[0829] composition

[0830] This system consists of a user device, a server, a generative AI model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative AI model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[0831] Data format conversion

[0832] In data format conversion, the user selects the required data in calculation software (e.g., Excel) and copies it to the clipboard. The terminal retrieves the data from the clipboard and sends it to the server. The server analyzes the received data and converts it into SQL format (e.g., INSERT INTO Sales (Year, Revenue, Expense) VALUES ...) using a generative AI model. The converted SQL data is sent back from the server to the terminal and displayed to the user. An example of an extracted prompt sentence is as follows:

[0833] Example prompt sentence:

[0834] Convert data containing columns such as "Year", "Revenue", and "Expense" into SQL format (e.g. INSERT INTO Sales (Year, Revenue, Expense) VALUES ...).

[0835] Inserting a line break

[0836] To insert a line break into SQL code, the user enters or pastes the SQL code on the terminal. The terminal sends the entered code to the server, which analyzes the received SQL code and uses a generative AI model to insert line breaks in the appropriate positions. The SQL code with line breaks is then sent back to the terminal and redisplayed to the user. An example of an extracted prompt sentence is as follows:

[0837] Example prompt sentence:

[0838] Insert line breaks at the appropriate positions in the following SQL code: Code: SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;

[0839] Emotion recognition by emotion engine

[0840] In emotion recognition using an emotion engine, the device uses a microphone and camera to collect the user's voice and facial expression data. The collected data is sent to the server in real time, where it is analyzed and used by the emotion engine to recognize the user's emotional state. For example, emotions are classified as positive, negative, or neutral. Based on the recognized emotional state, the server adjusts the feedback and generates detailed guidance and explanations. The generated feedback is sent to the device and displayed to the user. An example of an extracted prompt sentence is as follows:

[0841] Example prompt sentence:

[0842] Analyze voice and facial expression data to classify the user's emotional state (positive, negative, neutral, etc.). If it is negative, generate a detailed help message to provide feedback.

[0843] This allows users to quickly and efficiently convert data into specific formats, improve the readability of program code, and receive appropriate feedback through emotion recognition.

[0844] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0845] Data format conversion

[0846] Step 1:

[0847] The user opens a calculation software (e.g. Excel) and selects the columns "Year", "Sales", and "Expenses" to copy. The input of this step is the data selected by the user, and the output is the data copied to the clipboard.

[0848] Step 2:

[0849] The terminal gets data from the clipboard and sends it to the server. The input of this step is the data stored in the clipboard, and the output is the JSON formatted data that is sent to the server.

[0850] Step 3:

[0851] The server parses the received data and converts it into a specific SQL format using a generative AI model. The input of this step is JSON formatted data and the output is SQL formatted data.

[0852] Step 4:

[0853] The server sends the converted SQL data to the terminal as an HTTP response. The input of this step is data in SQL format, and the output is the data sent to the terminal as an HTTP response.

[0854] Step 5:

[0855] The terminal formats the received SQL data and displays it to the user. The input to this step is the SQL data received from the server, and the output is the formatted SQL data that is displayed to the user.

[0856] Inserting a line break

[0857] Step 1:

[0858] The user types SQL code into their terminal (e.g., "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000") or pastes it from the clipboard. The input for this step is the SQL code typed or pasted by the user, and the output is the SQL code displayed on the terminal.

[0859] Step 2:

[0860] The terminal sends the entered or pasted SQL code to the server. The input of this step is the SQL code entered or pasted by the user, and the output is the SQL code sent to the server.

[0861] Step 3:

[0862] The server parses the received SQL code and uses a generative AI model to insert line breaks at the appropriate positions. The input to this step is the received SQL code, and the output is the SQL code with line breaks inserted.

[0863] Step 4:

[0864] The server sends the SQL code with line breaks inserted to the terminal. The input of this step is the SQL code with line breaks inserted, and the output is the SQL code with line breaks sent to the terminal.

[0865] Step 5:

[0866] The terminal re-displays the received SQL code with line breaks to the user. The input of this step is the SQL code received from the server with line breaks, and the output is the SQL code with line breaks re-displayed to the user.

[0867] Emotion recognition by emotion engine

[0868] Step 1:

[0869] The device uses a microphone and a camera to collect the user's voice and facial expressions. The input of this step is the user's voice and facial expression data, and the output is the collected real-time emotion data.

[0870] Step 2:

[0871] The terminal transmits the collected emotion data to the server in real time. The input of this step is the collected emotion data, and the output is the emotion data transmitted to the server.

[0872] Step 3:

[0873] The server uses an emotion engine to recognize the user's emotional state from the transmitted voice and facial expression data. The input of this step is the collected voice and facial expression data, and the output is the recognized user's emotional state (e.g., positive, negative, neutral).

[0874] Step 4:

[0875] The server adjusts the feedback based on the recognized emotional state and generates detailed guides and explanations. The input of this step is the recognized emotional state and the output is the adjusted feedback data.

[0876] Step 5:

[0877] The server sends the generated feedback data to the terminal. The input of this step is the generated feedback data, and the output is the feedback data sent to the terminal.

[0878] Step 6:

[0879] The terminal displays the feedback received from the server to the user. The input of this step is the feedback data received from the server, and the output is the feedback displayed to the user.

[0880] (Application example 2)

[0881] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0882] Conventional data processing systems have limitations in converting user-specified data sets and improving the readability of program code. It is also difficult for systems to recognize users' emotional states and adjust feedback in real time, which is particularly important for improving the quality of customer service in brick-and-mortar stores. The present invention aims to solve these problems and significantly improve the user experience.

[0883] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative model to convert the dataset into a specific data format, means for providing the converted dataset to the user, means for using an emotion engine to recognize the emotional state of the user, and means for adjusting feedback based on the recognized emotional state. This enables the system to quickly and efficiently convert into a data format, improve the readability of program code, and adjust feedback based on emotion recognition.

[0884] A "dataset" refers to a series of data items or information specified by a user.

[0885] A "generative model" is a model that uses machine learning algorithms to transform or reorganize data into a specific format.

[0886] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state from input data such as voice and facial expressions and recognizes that state.

[0887] "Feedback" refers to the responses or instructions provided by a system based on a user's input or emotional state.

[0888] "Server" refers to a computer system that acts as a data processing and transformation center.

[0889] The system for implementing this invention consists of four main components: a server, a user terminal, a generative model, and an emotion engine. Below, we will provide an overview of the functions and processing of each of these components.

[0890] System Configuration

[0891] The system has the ability to receive user-specified data sets and convert them into specific data formats, as well as to insert line breaks at appropriate positions in user-entered program code and to recognize the user's emotional state and adjust feedback accordingly.

[0892] User terminal

[0893] The user terminal can use smart glasses and function as an interface for data input, reception, and transmission. This allows the user's voice and facial expressions to be collected in real time and transmitted to the server. The server also displays feedback to the user.

[0894] server

[0895] The server acts as a data processing and transformation center, using a generative model to transform the data and returning the converted dataset to the user in a specific data format. It also uses an emotion engine to recognize the user's emotional state and generate feedback based on that.

[0896] Generative Model

[0897] Generative models are used to transform data into specific formats or to insert line breaks into program code. They use machine learning algorithms to transform input data efficiently and accurately.

[0898] Emotion Engine

[0899] The emotion engine analyzes and recognizes the user's emotional state from input data such as voice and facial expressions, and if the emotional state is negative, it provides the user with detailed guidance and support messages.

[0900] System Processing Overview

[0901] Data format conversion

[0902] 1. Receiving a user-specified dataset

[0903] The user inputs a dataset using voice commands through the smart glasses. The device receives the data and prepares it for transmission to the server.

[0904] 2. Data format conversion by the server

[0905] The server uses the generative model to transform the received dataset into a specific format, for example, into an SQL query or a specific database format.

[0906] Emotion Recognition and Feedback

[0907] 1. Recognizing the user's emotional state

[0908] The smart glasses use a built-in camera and microphone to collect the user's voice and facial expressions, and the server uses an emotion engine to recognize the user's emotional state from these data.

[0909] 2. Adjusting feedback

[0910] The server adjusts the feedback based on the perceived emotional state, providing detailed help messages and direct support if negative emotions are detected.

[0911] Specific examples

[0912] For example, when a user voice-inputs a customer's order details through the smart glasses, the data set is sent from the smart glasses to the server and converted into an SQL query format. If the user's facial expression is confused, the server's emotion engine will recognize it and display a detailed guide message on the smart glasses.

[0913] Prompt Sentence Examples

[0914] Get customer order details and convert them into SQL format. Also, recognize the customer's emotional state and provide a support message if necessary.

[0915] This system is expected to improve customer service efficiency in physical stores and increase customer satisfaction.

[0916] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0917] Step 1:

[0918] The user specifies the customer's order by voice input via the smart glasses. The microphone in the smart glasses captures the voice data and converts it into text data using voice recognition software (e.g., Google Cloud Speech-to-Text). This text data is then input into the user's device.

[0919] Step 2:

[0920] The user device packages the acquired text data in JSON format and sends it to the server using an HTTP request. The server receives the request and parses the data using a JSON parser (e.g., json.loads).

[0921] Step 3:

[0922] The server inputs the analyzed data into a generative model (e.g., GPT-3) and performs data processing to convert it into a specific data format (e.g., SQL query). The server receives the SQL query output by the generative model.

[0923] Step 4:

[0924] The server sends the generated SQL query as an HTTP response to the user terminal, which receives the HTTP response and displays the acquired SQL query on the display of the smart glasses.

[0925] Step 5:

[0926] The smart glasses' camera and microphone are used to collect the customer's voice and facial expressions in real time, and this data is then input into the device, which then sends it to the server as an HTTP request.

[0927] Step 6:

[0928] The server inputs the received voice and facial expression data into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze and recognize the customer's emotional state. The emotion engine outputs the emotion recognition results to the server.

[0929] Step 7:

[0930] The server generates feedback based on the recognized emotional state. For example, if the customer is confused, it generates a detailed guide message. The server sends this feedback to the terminal as an HTTP response.

[0931] Step 8:

[0932] The user terminal displays the feedback received from the server on the display of the smart glasses, for example, a message such as "What's the matter? We'll show you around."

[0933] This series of processes is expected to significantly improve the quality and efficiency of customer service.

[0934] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0935] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0936] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0937] [Third embodiment]

[0938] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0939] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0940] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0941] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0942] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0943] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0944] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0945] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0946] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0947] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0948] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0949] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0950] The present invention provides a system that improves the efficiency of data processing tasks performed by users through calculation software. Specifically, it provides technology that automates data format conversion and line break insertion in program code, focusing on data processing devices.

[0951] composition

[0952] This system consists of a user terminal, a server, and a generative model. The user terminal serves as the interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques.

[0953] What the program does

[0954] Data format conversion

[0955] 1. Data selection and acquisition

[0956] The user selects the required data using calculation software (e.g. Excel), and the data is then copied.

[0957] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[0958] 2. Data transmission and conversion

[0959] The terminal transmits the acquired data to the server.

[0960] The server parses the received data and converts it into a specific SQL format using a generative model, which includes extracting column names and generating the appropriate SQL syntax.

[0961] 3. Return and display of converted data

[0962] The server returns the converted SQL data to the terminal.

[0963] The terminal displays the received SQL data to the user, who can then check it and use it as an SQL query.

[0964] Inserting a line break

[0965] 1. Enter and submit SQL code

[0966] The user types or copies and pastes the SQL code into the terminal.

[0967] The terminal sends the entered code to the server.

[0968] 2. Line break handling

[0969] The server parses the received SQL code and uses a generative model to insert line breaks at appropriate places, a process that automatically determines where to break lines to improve readability.

[0970] 3. Return and display of code after line break

[0971] The server returns the SQL code with line breaks inserted to the terminal.

[0972] The terminal displays the received SQL code with line breaks to the user, allowing the user to check the SQL code with improved readability.

[0973] Specific examples

[0974] Data format conversion example

[0975] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[0976] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0977] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[0978] Example of inserting a line break

[0979] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[0980] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[0981] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[0982] The configuration and processing of the present invention allow the user to quickly and efficiently convert data into a specific data format and improve the readability of program code.

[0983] The processing flow will be explained below.

[0984] Data format conversion

[0985] Step 1:

[0986] The user selects the necessary data in calculation software (e.g., Excel). For example, the user selects the range of columns "Fiscal Year," "Sales," and "Expenses" in an Excel sheet.

[0987] Step 2:

[0988] Copies the data selected by the user. The selected data is transferred to the clipboard.

[0989] Step 3:

[0990] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[0991] Step 4:

[0992] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[0993] Step 5:

[0994] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[0995] Step 6:

[0996] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[0997] Step 7:

[0998] The server sends the converted SQL data to the terminal as an HTTP response.

[0999] Step 8:

[1000] The terminal displays the received SQL data to the user in a dedicated window or message box.

[1001] Step 9:

[1002] The user checks the displayed SQL data and pastes it into a database for use.

[1003] Inserting a line break

[1004] Step 1:

[1005] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1006] Step 2:

[1007] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[1008] Step 3:

[1009] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[1010] Step 4:

[1011] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[1012] Step 5:

[1013] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[1014] Step 6:

[1015] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[1016] Step 7:

[1017] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[1018] Step 8:

[1019] The user can review the displayed SQL code and edit or use it as needed.

[1020] The above are the specific processing steps for data format conversion and line break insertion in this system.

[1021] Example 1

[1022] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1023] Conventional data processing devices require users to manually convert data into a specific format, which requires time and effort. Furthermore, manually inserting line breaks to improve program code readability is cumbersome and inefficient. This reduces user work efficiency and increases the likelihood of errors.

[1024] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1025] In this invention, the server includes a means for receiving a dataset specified by a user, a means for using a generative AI model to convert the dataset into a specific data format, and a means for providing the converted dataset to the user. This allows the user to convert data formats quickly and accurately. The server also includes a means for using a generative AI model to insert line breaks at appropriate positions in input program code, and a means for returning the code after the line breaks to the user terminal. This improves the readability of the program code and further enhances the user's work efficiency.

[1026] "User" means a person or entity that uses the system to process data or manipulate program code.

[1027] A "dataset" is a set of data that is subject to processing or transformation and may contain data items in different formats.

[1028] A "generative AI model" is a collection of algorithms that use machine learning techniques to automate specific tasks (such as converting data formats or inserting line breaks into code).

[1029] "Receiving means" refers to means for transmitting a data set or program code designated by a user to the system, and includes network communication and the like.

[1030] A "transformation means" is a means that performs a procedure or process to convert a received dataset into a particular format, typically using a generative AI model.

[1031] The "provision means" is a means for presenting the converted results to the user, and is carried out through a display device or a communication device.

[1032] The "clipboard" is a temporary data storage area that the user uses for copy and paste operations.

[1033] A "server" is a computer system that receives, analyzes, transforms, and returns data and has the processing power to run a generative AI model.

[1034] A "terminal" is a computer device that is directly operated by a user, and is a device for inputting data and displaying results.

[1035] "SQL format" refers to the format of instructions for performing database operations using Structured Query Language (SQL).

[1036] The present invention is a system that improves the efficiency of data processing tasks performed by users through calculation software. This system is configured around a data processing device and provides technology to automate conversion to specific data formats and line break insertion in program code.

[1037] composition

[1038] This system consists of a user terminal, a server, and a generative AI model. The roles and usage of each are explained below.

[1039] User terminal

[1040] A user device is an interface through which users input, receive, and send data. Specific hardware examples include personal computers (PCs), laptops, and tablets. Software examples include calculation software (e.g., Excel) and text editors (e.g., Visual Studio Code).

[1041] server

[1042] The server is the center of data processing and transformation. It analyzes the received data, converts it into SQL format, and inserts line breaks into the program code. The server is a computer system with a high-performance processor and large memory, and dedicated software for running the generative AI model is installed on it.

[1043] Generative AI Models

[1044] A generative AI model is an algorithm that uses machine learning techniques to automate specific data processing tasks. The model resides on a server and performs processes such as converting data formats and inserting line breaks into program code. Specifically, it applies natural language processing and data analysis techniques.

[1045] Data format conversion

[1046] 1. The user uses a calculation software (e.g. Excel) to select and copy the necessary data. For example, they select the columns "Fiscal Year," "Sales," and "Expenses."

[1047] 2. The device retrieves the selected data from the clipboard and prepares it to be sent to the server.

[1048] 3. The server parses the received data and uses a generative AI model to convert the data into a specific SQL format, for example, "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1049] 4. The server returns the converted SQL data to the terminal, which displays it to the user. The user can then check the SQL data and paste it into a database for use.

[1050] Inserting a line break

[1051] 1. A user types SQL code on a terminal or copies and pastes it from another source. For example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000" is entered.

[1052] 2. The device sends the entered code to the server.

[1053] 3. The server parses the received SQL code and uses a generative AI model to insert line breaks in the appropriate places, for example, "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1054] 4. The server returns the SQL code with line breaks to the terminal, which displays it to the user, who can then see the more readable SQL code and use it.

[1055] Specific examples

[1056] Data format conversion example

[1057] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[1058] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1059] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[1060] Example of inserting a line break

[1061] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1062] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1063] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[1064] Prompt Sentence Examples

[1065] "Convert the data selected in Excel into an SQL INSERT statement."

[1066] "Please insert appropriate line breaks into long SQL code."

[1067] As a result of the above, the system of the present invention can improve the efficiency of the user's data processing work and reduce the occurrence of errors.

[1068] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1069] Processing steps in data format conversion

[1070] Step 1:

[1071] The user opens a calculation software (e.g. Excel), selects the columns of data to be converted (e.g. "Fiscal Year", "Sales", "Expenses") and copies them. This operation saves the selected data to the user's clipboard.

[1072] Step 2:

[1073] The device retrieves data from the user's clipboard. This data is temporarily stored in internal memory, where it is analyzed for format and structure to verify that the data was retrieved correctly.

[1074] Step 3:

[1075] To send the data retrieved from the clipboard to the server, the device packages the data as an HTTP request, typically in JSON format, including the data cue.

[1076] Step 4:

[1077] The server receives the data sent from the device, analyzes the input data, and converts it into the appropriate format for input to the generative AI model. Specifically, it identifies the column names ("Fiscal Year," "Sales," "Expenses") and their corresponding data.

[1078] Step 5:

[1079] The server uses a generative AI model to convert the incoming data into SQL format, for example, generating SQL syntax like "INSERT INTO Sales (Year, Revenue, Expense) VALUES ..." The generative AI model creates the appropriate SQL statement, taking into account column names and data types.

[1080] Step 6:

[1081] The server returns the converted SQL data to the terminal as structured data in JSON format, etc. At this time, the data is sent using a communication protocol (e.g., HTTP).

[1082] Step 7:

[1083] The terminal receives the SQL data returned from the server, interprets it, and displays it to the user. It also formats it as SQL code and displays it in a text editor or browser in a format that is easy for the user to understand.

[1084] Processing steps for inserting line breaks

[1085] Step 1:

[1086] Users enter SQL code into a text editor on their terminal or copy and paste it from another source, for example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1087] Step 2:

[1088] The terminal temporarily stores the entered SQL code in its internal memory, analyzes this code to verify the integrity of the data, and then sends this code to the server.

[1089] Step 3:

[1090] To send the SQL code to the server, the terminal packages the data as an HTTP request, which includes the entire SQL code entered.

[1091] Step 4:

[1092] The server receives the SQL code sent from the device, analyzes the input data, and converts it into an appropriate format for input to the generative AI model. Specifically, it analyzes the grammar and keywords of the SQL statement.

[1093] Step 5:

[1094] The server uses a generative AI model to insert line breaks in the appropriate places, for example, transforming code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;" into "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1095] Step 6:

[1096] The server returns the SQL code after the line break to the terminal as structured data in JSON format or other format, using a communication protocol (e.g. HTTP) to send the data.

[1097] Step 7:

[1098] The terminal receives the SQL code returned from the server, interprets it, and displays it to the user. The SQL code is formatted and displayed in a readable format in a text editor or browser.

[1099] (Application example 1)

[1100] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1101] In modern factory automation, the format conversion of collected data and the management of complex program code are not efficiently carried out. This results in problems such as time-consuming data processing and reduced work efficiency. Furthermore, improving the readability of SQL code requires manual insertion of line breaks, which wastes human resources.

[1102] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1103] In this invention, the server includes a means for centrally managing data related to factory automation, a means for automatically converting the data into a required data format, and a means for inserting line breaks into the SQL code to improve readability, thereby making it possible to improve the efficiency of data processing work for factory automation and automate the management of program code.

[1104] A "user terminal" is a device that specifies a data set and provides a receiving and transmitting interface.

[1105] A "dataset" is a user-specified set of data that is to be converted into a particular format.

[1106] A "specific data format" is a format that is compatible with the calculation software or database query used by the user.

[1107] A "generative model" is a machine learning technique used to convert data formats and automatically format program code.

[1108] "Program code" is a series of instructions for executing calculation software or database operations.

[1109] "Inserting a line break" refers to the process of inserting a line break at an appropriate position in the program code to improve readability.

[1110] A "system" is a data management and processing mechanism consisting of a series of data processing devices, user terminals, servers, and generative models.

[1111] "Factory automation" refers to the equipment and devices used to automate tasks within a factory.

[1112] "Centralized management" refers to collecting and managing data in one place, and providing unified access and operation.

[1113] "Processing efficiently" refers to processing data or program code quickly and accurately.

[1114] "SQL code" means statements in a structured query language used to manipulate relational databases.

[1115] To implement this invention, a system consisting of a user terminal, a server, and a generative model is used. The user terminal works in conjunction with the control system of factory robots and industrial equipment, providing an interface for specifying datasets. The server functions as the center of data processing and conversion, converting data into a specific data format using the generative model.

[1116] The user device receives data collected from the factory robots and sends it to a cloud server. This process connects with calculation software and databases that are convenient for managing and visualizing the data. The server analyzes the received dataset and uses a generative model to convert it into a specific data format, such as SQL. The converted dataset is then sent back to the user device, where the user can use it as a database query.

[1117] In a specific implementation, a user first selects the necessary data using the factory robot control system. The selected data is then sent to a server via the cloud, where it is automatically converted into a specific data format using a generative model. For example, factory work data collected in Excel format can be converted into SQL format and returned to the user's device. The server can also automatically insert line breaks in entered SQL code and other program code at appropriate positions to improve readability.

[1118] The required hardware is a user terminal (a factory robot control unit or industrial PC) and a server (a cloud-based data processing server). The software uses the Pandas library, which is useful for data management, and the requests library, which processes HTTP requests. A machine learning model (e.g., GPT-3) is used as the generative model.

[1119] Specific examples are shown below.

[1120] For example, enter the following as the prompt:

[1121] Prompt: Convert the following Excel data into the appropriate SQL format.

[1122] data:

[1123] Year:2023, 2024

[1124] Sales: 10000, 15000

[1125] Cost: 5000, 7000

[1126] Expected output:

[1127] INSERT INTO Sales (Year, Revenue, Expense) VALUES (2023, 10000, 5000), (2024, 15000, 7000);

[1128] This allows for efficient data format conversion and program code processing related to factory automation, thereby improving work efficiency.

[1129] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1130] Step 1:

[1131] The user selects and specifies the required data using the factory robot control system.

[1132] Input: Work data collected from factory robots.

[1133] Output: A dataset specified by the user's device.

[1134] Specific operation: The user selects and copies data in Excel format, etc.

[1135] Step 2:

[1136] The device retrieves the specified data set from the clipboard and prepares it to be sent to the server.

[1137] Input: The dataset copied to the clipboard.

[1138] Output: The dataset that is sent to the server.

[1139] Specific operation: Retrieves data from the clipboard, converts it to JSON format, and prepares it to be sent to the server.

[1140] Step 3:

[1141] The server receives the dataset sent from the device and converts it into a specific data format using a generative model.

[1142] Input: Dataset submitted in JSON format.

[1143] Output: A dataset converted into a specific data format (e.g., SQL format).

[1144] Specific operation: The server receives the dataset and converts it into SQL format using a generative model (e.g., GPT-3).

[1145] Step 4:

[1146] The server returns the transformed data set to the user terminal.

[1147] Input: A dataset converted into a specific data format.

[1148] Output: The dataset that is sent back to the user's device.

[1149] Specific operation: The converted dataset is sent to the user terminal as a response to the HTTP request.

[1150] Step 5:

[1151] The terminal provides the received converted data to the user and displays it.

[1152] Input: A dataset returned from the server in a specific data format.

[1153] Output: The data format, such as an SQL query, that is made available to the user.

[1154] Specific operation: The converted dataset is displayed on the terminal, and the user pastes it into the database for use.

[1155] Step 6:

[1156] Requires the user to type or copy and paste program code (e.g., SQL code) and insert line breaks at the appropriate places.

[1157] Input: Program code typed or pasted by the user.

[1158] Output: The program code sent from the terminal to the server.

[1159] What it does: Copies the code the user types or pastes to the clipboard and prepares it to be sent to the server.

[1160] Step 7:

[1161] The server analyzes the received program code and uses a generative model to insert line breaks at the appropriate positions.

[1162] Input: The program code sent from the terminal.

[1163] Output: Program code with line breaks inserted.

[1164] Specific operation: The code received by the server is analyzed using a generative model, and line breaks are automatically determined and inserted to improve readability.

[1165] Step 8:

[1166] The server returns the program code with line breaks inserted to the terminal.

[1167] Input: Program code with line breaks inserted.

[1168] Output: Program code returned to the terminal with line breaks.

[1169] Specific operation: The program code after the line break is sent to the terminal as a response to the HTTP request.

[1170] Step 9:

[1171] The terminal displays the program code after the line break to the user.

[1172] Input: The program code returned from the server after the line break.

[1173] Output: Readable program code available to the user.

[1174] Specific operation: The program code after line breaks is displayed on the terminal so that the user can use it as is.

[1175] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1176] This invention relates to a system that not only improves the efficiency of data processing work using computer software, but also provides a feedback adjustment function based on user emotion recognition. Specifically, it is composed of a data processing device that converts data formats, inserts line breaks into program code, and recognizes emotions using an emotion engine.

[1177] composition

[1178] This system consists of a user device, a server, a generative model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[1179] What the program does

[1180] Data format conversion

[1181] 1. Data selection and acquisition

[1182] The user selects the required data in a calculation software (e.g. Excel). After selection, the data is copied.

[1183] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[1184] 2. Data transmission and conversion

[1185] The device sends the acquired data to the server via an HTTP request. The data is packaged in JSON format or similar.

[1186] The server parses the received data and converts it into a specific SQL format using a generative model.

[1187] 3. Return and display of converted data

[1188] The server sends the converted SQL data to the terminal as an HTTP response.

[1189] The terminal displays the received SQL data to the user.

[1190] Inserting a line break

[1191] 1. Enter and submit SQL code

[1192] The user types or pastes SQL code into the terminal.

[1193] The terminal sends the entered code to the server.

[1194] 2. Line break handling

[1195] The server parses the received SQL code and uses the generative model to insert line breaks at the appropriate places.

[1196] 3. Returning and displaying the code after the line break

[1197] The server sends the SQL code with line breaks inserted to the terminal.

[1198] The terminal re-displays the received SQL code after line breaks in the input field.

[1199] Emotion recognition by emotion engine

[1200] 1. Collecting Emotional Data

[1201] The device uses a microphone and camera to collect the user's voice and facial expressions.

[1202] 2. Emotion Recognition Analysis

[1203] The server uses an emotion engine to recognize the user's emotional state from the collected voice and facial expression data, for example classifying emotions as positive, negative, neutral, etc.

[1204] 3. Adjusting feedback

[1205] The server adjusts the feedback of the data processing based on the recognized emotion, for example, providing more detailed explanations or guidance if the user is showing negative emotions.

[1206] 4. Providing Feedback

[1207] The terminal provides the user with the feedback received from the server, thereby improving the user's work efficiency and satisfaction.

[1208] Specific examples

[1209] Data format conversion example

[1210] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[1211] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1212] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[1213] Example of inserting a line break

[1214] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1215] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1216] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[1217] Example of feedback adjustment using emotion recognition

[1218] While the user is using the system, the terminal collects the user's voice and facial expressions.

[1219] The server uses an emotion engine to recognize that the user is confused.

[1220] The server generates a more detailed help message and sends it to the terminal.

[1221] The terminal displays a help message to the user, and the user receives assistance in resolving the problem.

[1222] The configuration and processing of the present invention allow users to quickly and efficiently convert data into specific data formats and improve the readability of program code, and also provide a better user experience through feedback adjustment based on emotion recognition.

[1223] The processing flow will be explained below.

[1224] Data format conversion

[1225] Step 1:

[1226] The user selects the required data in calculation software (e.g. Excel). For example, the user selects a range of columns such as "Fiscal Year," "Sales," and "Expenses."

[1227] Step 2:

[1228] Copies the data selected by the user. The selected data is transferred to the clipboard.

[1229] Step 3:

[1230] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[1231] Step 4:

[1232] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[1233] Step 5:

[1234] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[1235] Step 6:

[1236] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1237] Step 7:

[1238] The server sends the converted SQL data to the terminal as an HTTP response.

[1239] Step 8:

[1240] The terminal displays the received SQL data to the user in a dedicated window or message box.

[1241] Step 9:

[1242] The user checks the displayed SQL data and pastes it into a database for use.

[1243] Inserting a line break

[1244] Step 1:

[1245] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1246] Step 2:

[1247] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[1248] Step 3:

[1249] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[1250] Step 4:

[1251] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[1252] Step 5:

[1253] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[1254] Step 6:

[1255] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[1256] Step 7:

[1257] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[1258] Step 8:

[1259] The user can review the displayed SQL code and edit or use it as needed.

[1260] Emotion recognition and feedback adjustment by emotion engine

[1261] Step 1:

[1262] The device collects the user's voice and facial expression data in real time using a microphone and camera, for example, capturing the user's face with a webcam.

[1263] Step 2:

[1264] The device will transmit the collected voice and facial expression data to a server, using appropriate data security techniques.

[1265] Step 3:

[1266] The server uses an emotion engine to analyze the user's voice and facial expression data, for example, by evaluating voice tone and facial muscle movements to identify emotions.

[1267] Step 4:

[1268] The server adjusts the feedback of the data processing based on the perceived emotion, for example generating a more detailed help message if it determines that the user is confused.

[1269] Step 5:

[1270] Sends server-generated feedback to the device, which may include additional explanations or guidance.

[1271] Step 6:

[1272] The device displays the received feedback to the user, allowing the user to work more efficiently.

[1273] The above are the specific processing steps for data format conversion, line break insertion, and feedback adjustment through emotion recognition in this system.

[1274] Example 2

[1275] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1276] In conventional data processing systems, the format conversion of user-specified data sets and the formatting of program code are often performed manually, which reduces efficiency and leads to errors.In addition, one-sided feedback is provided without considering the user's emotional state, which leads to a lack of user satisfaction.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1278] In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative AI model to convert the dataset into a specific data format, means for acquiring data from a clipboard, means for providing the converted dataset to the user, means for collecting voice and facial expressions to recognize the emotional state of the user, and means for recognizing emotions from the collected data using an emotion engine and adjusting feedback based on the recognized emotions. This allows for efficient automatic data conversion and program code formatting, and further enables appropriate feedback to be provided based on the user's emotional state.

[1279] A "dataset" is a set of data that a user specifies they want to process.

[1280] A "generative AI model" is an algorithm that uses machine learning techniques to generate specific outputs based on large amounts of data.

[1281] The "clipboard" is a memory area that temporarily stores data within a terminal and enables data transfer between other applications.

[1282] "Emotion engine" is a general term for algorithms and software that analyze voice and facial expression data to identify a user's emotional state.

[1283] "Feedback" refers to the reactions and information provided by a system in response to a user's actions and emotions.

[1284] A "user" is a person or entity that is responsible for operating the system and providing data and instructions.

[1285] The present invention is a system that utilizes computer software to provide efficient data processing and feedback adjustment based on user emotion recognition. Specifically, it constitutes a data processing device that includes data format conversion, line break insertion in program code, and emotion recognition using an emotion engine.

[1286] composition

[1287] This system consists of a user device, a server, a generative AI model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative AI model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[1288] Data format conversion

[1289] In data format conversion, the user selects the required data in calculation software (e.g., Excel) and copies it to the clipboard. The terminal retrieves the data from the clipboard and sends it to the server. The server analyzes the received data and converts it into SQL format (e.g., INSERT INTO Sales (Year, Revenue, Expense) VALUES ...) using a generative AI model. The converted SQL data is sent back from the server to the terminal and displayed to the user. An example of an extracted prompt sentence is as follows:

[1290] Example prompt sentence:

[1291] Convert data containing columns such as "Year", "Revenue", and "Expense" into SQL format (e.g. INSERT INTO Sales (Year, Revenue, Expense) VALUES ...).

[1292] Inserting a line break

[1293] To insert a line break into SQL code, the user enters or pastes the SQL code on the terminal. The terminal sends the entered code to the server, which analyzes the received SQL code and uses a generative AI model to insert line breaks in the appropriate positions. The SQL code with line breaks is then sent back to the terminal and redisplayed to the user. An example of an extracted prompt sentence is as follows:

[1294] Example prompt sentence:

[1295] Insert line breaks at the appropriate positions in the following SQL code: Code: SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;

[1296] Emotion recognition by emotion engine

[1297] In emotion recognition using an emotion engine, the device uses a microphone and camera to collect the user's voice and facial expression data. The collected data is sent to the server in real time, where it is analyzed and used by the emotion engine to recognize the user's emotional state. For example, emotions are classified as positive, negative, or neutral. Based on the recognized emotional state, the server adjusts the feedback and generates detailed guidance and explanations. The generated feedback is sent to the device and displayed to the user. An example of an extracted prompt sentence is as follows:

[1298] Example prompt sentence:

[1299] Analyze voice and facial expression data to classify the user's emotional state (positive, negative, neutral, etc.). If it is negative, generate a detailed help message to provide feedback.

[1300] This allows users to quickly and efficiently convert data into specific formats, improve the readability of program code, and receive appropriate feedback through emotion recognition.

[1301] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1302] Data format conversion

[1303] Step 1:

[1304] The user opens a calculation software (e.g. Excel) and selects the columns "Year", "Sales", and "Expenses" to copy. The input of this step is the data selected by the user, and the output is the data copied to the clipboard.

[1305] Step 2:

[1306] The terminal gets data from the clipboard and sends it to the server. The input of this step is the data stored in the clipboard, and the output is the JSON formatted data that is sent to the server.

[1307] Step 3:

[1308] The server parses the received data and converts it into a specific SQL format using a generative AI model. The input of this step is JSON formatted data and the output is SQL formatted data.

[1309] Step 4:

[1310] The server sends the converted SQL data to the terminal as an HTTP response. The input of this step is data in SQL format, and the output is the data sent to the terminal as an HTTP response.

[1311] Step 5:

[1312] The terminal formats the received SQL data and displays it to the user. The input to this step is the SQL data received from the server, and the output is the formatted SQL data that is displayed to the user.

[1313] Inserting a line break

[1314] Step 1:

[1315] The user types SQL code into their terminal (e.g., "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000") or pastes it from the clipboard. The input for this step is the SQL code typed or pasted by the user, and the output is the SQL code displayed on the terminal.

[1316] Step 2:

[1317] The terminal sends the entered or pasted SQL code to the server. The input of this step is the SQL code entered or pasted by the user, and the output is the SQL code sent to the server.

[1318] Step 3:

[1319] The server parses the received SQL code and uses a generative AI model to insert line breaks at the appropriate positions. The input to this step is the received SQL code, and the output is the SQL code with line breaks inserted.

[1320] Step 4:

[1321] The server sends the SQL code with line breaks inserted to the terminal. The input of this step is the SQL code with line breaks inserted, and the output is the SQL code with line breaks sent to the terminal.

[1322] Step 5:

[1323] The terminal re-displays the received SQL code with line breaks to the user. The input of this step is the SQL code received from the server with line breaks, and the output is the SQL code with line breaks re-displayed to the user.

[1324] Emotion recognition by emotion engine

[1325] Step 1:

[1326] The device uses a microphone and a camera to collect the user's voice and facial expressions. The input of this step is the user's voice and facial expression data, and the output is the collected real-time emotion data.

[1327] Step 2:

[1328] The terminal transmits the collected emotion data to the server in real time. The input of this step is the collected emotion data, and the output is the emotion data transmitted to the server.

[1329] Step 3:

[1330] The server uses an emotion engine to recognize the user's emotional state from the transmitted voice and facial expression data. The input of this step is the collected voice and facial expression data, and the output is the recognized user's emotional state (e.g., positive, negative, neutral).

[1331] Step 4:

[1332] The server adjusts the feedback based on the recognized emotional state and generates detailed guides and explanations. The input of this step is the recognized emotional state and the output is the adjusted feedback data.

[1333] Step 5:

[1334] The server sends the generated feedback data to the terminal. The input of this step is the generated feedback data, and the output is the feedback data sent to the terminal.

[1335] Step 6:

[1336] The terminal displays the feedback received from the server to the user. The input of this step is the feedback data received from the server, and the output is the feedback displayed to the user.

[1337] (Application example 2)

[1338] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1339] Conventional data processing systems have limitations in converting user-specified data sets and improving the readability of program code. It is also difficult for systems to recognize users' emotional states and adjust feedback in real time, which is particularly important for improving the quality of customer service in brick-and-mortar stores. The present invention aims to solve these problems and significantly improve the user experience.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative model to convert the dataset into a specific data format, means for providing the converted dataset to the user, means for using an emotion engine to recognize the emotional state of the user, and means for adjusting feedback based on the recognized emotional state. This enables the system to quickly and efficiently convert into a data format, improve the readability of program code, and adjust feedback based on emotion recognition.

[1341] A "dataset" refers to a series of data items or information specified by a user.

[1342] A "generative model" is a model that uses machine learning algorithms to transform or reorganize data into a specific format.

[1343] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state from input data such as voice and facial expressions and recognizes that state.

[1344] "Feedback" refers to the responses or instructions provided by a system based on a user's input or emotional state.

[1345] "Server" refers to a computer system that acts as a data processing and transformation center.

[1346] The system for implementing this invention consists of four main components: a server, a user terminal, a generative model, and an emotion engine. Below, we will provide an overview of the functions and processing of each of these components.

[1347] System Configuration

[1348] The system has the ability to receive user-specified data sets and convert them into specific data formats, as well as to insert line breaks at appropriate positions in user-entered program code and to recognize the user's emotional state and adjust feedback accordingly.

[1349] User terminal

[1350] The user terminal can use smart glasses and function as an interface for data input, reception, and transmission. This allows the user's voice and facial expressions to be collected in real time and transmitted to the server. The server also displays feedback to the user.

[1351] server

[1352] The server acts as a data processing and transformation center, using a generative model to transform the data and returning the converted dataset to the user in a specific data format. It also uses an emotion engine to recognize the user's emotional state and generate feedback based on that.

[1353] Generative Model

[1354] Generative models are used to transform data into specific formats or to insert line breaks into program code. They use machine learning algorithms to transform input data efficiently and accurately.

[1355] Emotion Engine

[1356] The emotion engine analyzes and recognizes the user's emotional state from input data such as voice and facial expressions, and if the emotional state is negative, it provides the user with detailed guidance and support messages.

[1357] System Processing Overview

[1358] Data format conversion

[1359] 1. Receiving a user-specified dataset

[1360] The user inputs a dataset using voice commands through the smart glasses. The device receives the data and prepares it for transmission to the server.

[1361] 2. Data format conversion by the server

[1362] The server uses the generative model to transform the received dataset into a specific format, for example, into an SQL query or a specific database format.

[1363] Emotion Recognition and Feedback

[1364] 1. Recognizing the user's emotional state

[1365] The smart glasses use a built-in camera and microphone to collect the user's voice and facial expressions, and the server uses an emotion engine to recognize the user's emotional state from these data.

[1366] 2. Adjusting feedback

[1367] The server adjusts the feedback based on the perceived emotional state, providing detailed help messages and direct support if negative emotions are detected.

[1368] Specific examples

[1369] For example, when a user voice-inputs a customer's order details through the smart glasses, the data set is sent from the smart glasses to the server and converted into an SQL query format. If the user's facial expression is confused, the server's emotion engine will recognize it and display a detailed guide message on the smart glasses.

[1370] Prompt Sentence Examples

[1371] Get customer order details and convert them into SQL format. Also, recognize the customer's emotional state and provide a support message if necessary.

[1372] This system is expected to improve customer service efficiency in physical stores and increase customer satisfaction.

[1373] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1374] Step 1:

[1375] The user specifies the customer's order by voice input via the smart glasses. The microphone in the smart glasses captures the voice data and converts it into text data using voice recognition software (e.g., Google Cloud Speech-to-Text). This text data is then input into the user's device.

[1376] Step 2:

[1377] The user device packages the acquired text data in JSON format and sends it to the server using an HTTP request. The server receives the request and parses the data using a JSON parser (e.g., json.loads).

[1378] Step 3:

[1379] The server inputs the analyzed data into a generative model (e.g., GPT-3) and performs data processing to convert it into a specific data format (e.g., SQL query). The server receives the SQL query output by the generative model.

[1380] Step 4:

[1381] The server sends the generated SQL query as an HTTP response to the user terminal, which receives the HTTP response and displays the acquired SQL query on the display of the smart glasses.

[1382] Step 5:

[1383] The smart glasses' camera and microphone are used to collect the customer's voice and facial expressions in real time, and this data is then input into the device, which then sends it to the server as an HTTP request.

[1384] Step 6:

[1385] The server inputs the received voice and facial expression data into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze and recognize the customer's emotional state. The emotion engine outputs the emotion recognition results to the server.

[1386] Step 7:

[1387] The server generates feedback based on the recognized emotional state. For example, if the customer is confused, it generates a detailed guide message. The server sends this feedback to the terminal as an HTTP response.

[1388] Step 8:

[1389] The user terminal displays the feedback received from the server on the display of the smart glasses, for example, a message such as "What's the matter? We'll show you around."

[1390] This series of processes is expected to significantly improve the quality and efficiency of customer service.

[1391] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1392] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1393] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1394] [Fourth embodiment]

[1395] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1396] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1397] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1398] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1399] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1400] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1401] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1402] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1403] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1404] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1405] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1406] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1407] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1408] The present invention provides a system that improves the efficiency of data processing tasks performed by users through calculation software. Specifically, it provides technology that automates data format conversion and line break insertion in program code, focusing on data processing devices.

[1409] composition

[1410] This system consists of a user terminal, a server, and a generative model. The user terminal serves as the interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques.

[1411] What the program does

[1412] Data format conversion

[1413] 1. Data selection and acquisition

[1414] The user selects the required data using calculation software (e.g. Excel), and the data is then copied.

[1415] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[1416] 2. Data transmission and conversion

[1417] The terminal transmits the acquired data to the server.

[1418] The server parses the received data and converts it into a specific SQL format using a generative model, which includes extracting column names and generating the appropriate SQL syntax.

[1419] 3. Return and display of converted data

[1420] The server returns the converted SQL data to the terminal.

[1421] The terminal displays the received SQL data to the user, who can then check it and use it as an SQL query.

[1422] Inserting a line break

[1423] 1. Enter and submit SQL code

[1424] The user types or copies and pastes the SQL code into the terminal.

[1425] The terminal sends the entered code to the server.

[1426] 2. Line break handling

[1427] The server parses the received SQL code and uses a generative model to insert line breaks at appropriate places, a process that automatically determines where to break lines to improve readability.

[1428] 3. Return and display of code after line break

[1429] The server returns the SQL code with line breaks inserted to the terminal.

[1430] The terminal displays the received SQL code with line breaks to the user, allowing the user to check the SQL code with improved readability.

[1431] Specific examples

[1432] Data format conversion example

[1433] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[1434] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1435] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[1436] Example of inserting a line break

[1437] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1438] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1439] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[1440] The configuration and processing of the present invention allow the user to quickly and efficiently convert data into a specific data format and improve the readability of program code.

[1441] The processing flow will be explained below.

[1442] Data format conversion

[1443] Step 1:

[1444] The user selects the necessary data in calculation software (e.g., Excel). For example, the user selects the range of columns "Fiscal Year," "Sales," and "Expenses" in an Excel sheet.

[1445] Step 2:

[1446] Copies the data selected by the user. The selected data is transferred to the clipboard.

[1447] Step 3:

[1448] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[1449] Step 4:

[1450] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[1451] Step 5:

[1452] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[1453] Step 6:

[1454] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1455] Step 7:

[1456] The server sends the converted SQL data to the terminal as an HTTP response.

[1457] Step 8:

[1458] The terminal displays the received SQL data to the user in a dedicated window or message box.

[1459] Step 9:

[1460] The user checks the displayed SQL data and pastes it into a database for use.

[1461] Inserting a line break

[1462] Step 1:

[1463] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1464] Step 2:

[1465] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[1466] Step 3:

[1467] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[1468] Step 4:

[1469] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[1470] Step 5:

[1471] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[1472] Step 6:

[1473] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[1474] Step 7:

[1475] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[1476] Step 8:

[1477] The user can review the displayed SQL code and edit or use it as needed.

[1478] The above are the specific processing steps for data format conversion and line break insertion in this system.

[1479] Example 1

[1480] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1481] Conventional data processing devices require users to manually convert data into a specific format, which requires time and effort. Furthermore, manually inserting line breaks to improve program code readability is cumbersome and inefficient. This reduces user work efficiency and increases the likelihood of errors.

[1482] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1483] In this invention, the server includes a means for receiving a dataset specified by a user, a means for using a generative AI model to convert the dataset into a specific data format, and a means for providing the converted dataset to the user. This allows the user to convert data formats quickly and accurately. The server also includes a means for using a generative AI model to insert line breaks at appropriate positions in input program code, and a means for returning the code after the line breaks to the user terminal. This improves the readability of the program code and further enhances the user's work efficiency.

[1484] "User" means a person or entity that uses the system to process data or manipulate program code.

[1485] A "dataset" is a set of data that is subject to processing or transformation and may contain data items in different formats.

[1486] A "generative AI model" is a collection of algorithms that use machine learning techniques to automate specific tasks (such as converting data formats or inserting line breaks into code).

[1487] "Receiving means" refers to means for transmitting a data set or program code designated by a user to the system, and includes network communication and the like.

[1488] A "transformation means" is a means that performs a procedure or process to convert a received dataset into a particular format, typically using a generative AI model.

[1489] The "provision means" is a means for presenting the converted results to the user, and is carried out through a display device or a communication device.

[1490] The "clipboard" is a temporary data storage area that the user uses for copy and paste operations.

[1491] A "server" is a computer system that receives, analyzes, transforms, and returns data and has the processing power to run a generative AI model.

[1492] A "terminal" is a computer device that is directly operated by a user, and is a device for inputting data and displaying results.

[1493] "SQL format" refers to the format of instructions for performing database operations using Structured Query Language (SQL).

[1494] The present invention is a system that improves the efficiency of data processing tasks performed by users through calculation software. This system is configured around a data processing device and provides technology to automate conversion to specific data formats and line break insertion in program code.

[1495] composition

[1496] This system consists of a user terminal, a server, and a generative AI model. The roles and usage of each are explained below.

[1497] User terminal

[1498] A user device is an interface through which users input, receive, and send data. Specific hardware examples include personal computers (PCs), laptops, and tablets. Software examples include calculation software (e.g., Excel) and text editors (e.g., Visual Studio Code).

[1499] server

[1500] The server is the center of data processing and transformation. It analyzes the received data, converts it into SQL format, and inserts line breaks into the program code. The server is a computer system with a high-performance processor and large memory, and dedicated software for running the generative AI model is installed on it.

[1501] Generative AI Models

[1502] A generative AI model is an algorithm that uses machine learning techniques to automate specific data processing tasks. The model resides on a server and performs processes such as converting data formats and inserting line breaks into program code. Specifically, it applies natural language processing and data analysis techniques.

[1503] Data format conversion

[1504] 1. The user uses a calculation software (e.g. Excel) to select and copy the necessary data. For example, they select the columns "Fiscal Year," "Sales," and "Expenses."

[1505] 2. The device retrieves the selected data from the clipboard and prepares it to be sent to the server.

[1506] 3. The server parses the received data and uses a generative AI model to convert the data into a specific SQL format, for example, "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1507] 4. The server returns the converted SQL data to the terminal, which displays it to the user. The user can then check the SQL data and paste it into a database for use.

[1508] Inserting a line break

[1509] 1. A user types SQL code on a terminal or copies and pastes it from another source. For example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000" is entered.

[1510] 2. The device sends the entered code to the server.

[1511] 3. The server parses the received SQL code and uses a generative AI model to insert line breaks in the appropriate places, for example, "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1512] 4. The server returns the SQL code with line breaks to the terminal, which displays it to the user, who can then see the more readable SQL code and use it.

[1513] Specific examples

[1514] Data format conversion example

[1515] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[1516] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1517] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[1518] Example of inserting a line break

[1519] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1520] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1521] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[1522] Prompt Sentence Examples

[1523] "Convert the data selected in Excel into an SQL INSERT statement."

[1524] "Please insert appropriate line breaks into long SQL code."

[1525] As a result of the above, the system of the present invention can improve the efficiency of the user's data processing work and reduce the occurrence of errors.

[1526] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1527] Processing steps in data format conversion

[1528] Step 1:

[1529] The user opens a calculation software (e.g. Excel), selects the columns of data to be converted (e.g. "Fiscal Year", "Sales", "Expenses") and copies them. This operation saves the selected data to the user's clipboard.

[1530] Step 2:

[1531] The device retrieves data from the user's clipboard. This data is temporarily stored in internal memory, where it is analyzed for format and structure to verify that the data was retrieved correctly.

[1532] Step 3:

[1533] To send the data retrieved from the clipboard to the server, the device packages the data as an HTTP request, typically in JSON format, including the data cue.

[1534] Step 4:

[1535] The server receives the data sent from the device, analyzes the input data, and converts it into the appropriate format for input to the generative AI model. Specifically, it identifies the column names ("Fiscal Year," "Sales," "Expenses") and their corresponding data.

[1536] Step 5:

[1537] The server uses a generative AI model to convert the incoming data into SQL format, for example, generating SQL syntax like "INSERT INTO Sales (Year, Revenue, Expense) VALUES ..." The generative AI model creates the appropriate SQL statement, taking into account column names and data types.

[1538] Step 6:

[1539] The server returns the converted SQL data to the terminal as structured data in JSON format, etc. At this time, the data is sent using a communication protocol (e.g., HTTP).

[1540] Step 7:

[1541] The terminal receives the SQL data returned from the server, interprets it, and displays it to the user. It also formats it as SQL code and displays it in a text editor or browser in a format that is easy for the user to understand.

[1542] Processing steps for inserting line breaks

[1543] Step 1:

[1544] Users enter SQL code into a text editor on their terminal or copy and paste it from another source, for example, a long piece of SQL code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1545] Step 2:

[1546] The terminal temporarily stores the entered SQL code in its internal memory, analyzes this code to verify the integrity of the data, and then sends this code to the server.

[1547] Step 3:

[1548] To send the SQL code to the server, the terminal packages the data as an HTTP request, which includes the entire SQL code entered.

[1549] Step 4:

[1550] The server receives the SQL code sent from the device, analyzes the input data, and converts it into an appropriate format for input to the generative AI model. Specifically, it analyzes the grammar and keywords of the SQL statement.

[1551] Step 5:

[1552] The server uses a generative AI model to insert line breaks in the appropriate places, for example, transforming code like "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;" into "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1553] Step 6:

[1554] The server returns the SQL code after the line break to the terminal as structured data in JSON format or other format, using a communication protocol (e.g. HTTP) to send the data.

[1555] Step 7:

[1556] The terminal receives the SQL code returned from the server, interprets it, and displays it to the user. The SQL code is formatted and displayed in a readable format in a text editor or browser.

[1557] (Application example 1)

[1558] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1559] In modern factory automation, the format conversion of collected data and the management of complex program code are not efficiently carried out. This results in problems such as time-consuming data processing and reduced work efficiency. Furthermore, improving the readability of SQL code requires manual insertion of line breaks, which wastes human resources.

[1560] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1561] In this invention, the server includes a means for centrally managing data related to factory automation, a means for automatically converting the data into a required data format, and a means for inserting line breaks into the SQL code to improve readability, thereby making it possible to improve the efficiency of data processing work for factory automation and automate the management of program code.

[1562] A "user terminal" is a device that specifies a data set and provides a receiving and transmitting interface.

[1563] A "dataset" is a user-specified set of data that is to be converted into a particular format.

[1564] A "specific data format" is a format that is compatible with the calculation software or database query used by the user.

[1565] A "generative model" is a machine learning technique used to convert data formats and automatically format program code.

[1566] "Program code" is a series of instructions for executing calculation software or database operations.

[1567] "Inserting a line break" refers to the process of inserting a line break at an appropriate position in the program code to improve readability.

[1568] A "system" is a data management and processing mechanism consisting of a series of data processing devices, user terminals, servers, and generative models.

[1569] "Factory automation" refers to the equipment and devices used to automate tasks within a factory.

[1570] "Centralized management" refers to collecting and managing data in one place, and providing unified access and operation.

[1571] "Processing efficiently" refers to processing data or program code quickly and accurately.

[1572] "SQL code" means statements in a structured query language used to manipulate relational databases.

[1573] To implement this invention, a system consisting of a user terminal, a server, and a generative model is used. The user terminal works in conjunction with the control system of factory robots and industrial equipment, providing an interface for specifying datasets. The server functions as the center of data processing and conversion, converting data into a specific data format using the generative model.

[1574] The user device receives data collected from the factory robots and sends it to a cloud server. This process connects with calculation software and databases that are convenient for managing and visualizing the data. The server analyzes the received dataset and uses a generative model to convert it into a specific data format, such as SQL. The converted dataset is then sent back to the user device, where the user can use it as a database query.

[1575] In a specific implementation, a user first selects the necessary data using the factory robot control system. The selected data is then sent to a server via the cloud, where it is automatically converted into a specific data format using a generative model. For example, factory work data collected in Excel format can be converted into SQL format and returned to the user's device. The server can also automatically insert line breaks in entered SQL code and other program code at appropriate positions to improve readability.

[1576] The required hardware is a user terminal (a factory robot control unit or industrial PC) and a server (a cloud-based data processing server). The software uses the Pandas library, which is useful for data management, and the requests library, which processes HTTP requests. A machine learning model (e.g., GPT-3) is used as the generative model.

[1577] Specific examples are shown below.

[1578] For example, enter the following as the prompt:

[1579] Prompt: Convert the following Excel data into the appropriate SQL format.

[1580] data:

[1581] Year:2023, 2024

[1582] Sales: 10000, 15000

[1583] Cost: 5000, 7000

[1584] Expected output:

[1585] INSERT INTO Sales (Year, Revenue, Expense) VALUES (2023, 10000, 5000), (2024, 15000, 7000);

[1586] This allows for efficient data format conversion and program code processing related to factory automation, thereby improving work efficiency.

[1587] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1588] Step 1:

[1589] The user selects and specifies the required data using the factory robot control system.

[1590] Input: Work data collected from factory robots.

[1591] Output: A dataset specified by the user's device.

[1592] Specific operation: The user selects and copies data in Excel format, etc.

[1593] Step 2:

[1594] The device retrieves the specified data set from the clipboard and prepares it to be sent to the server.

[1595] Input: The dataset copied to the clipboard.

[1596] Output: The dataset that is sent to the server.

[1597] Specific operation: Retrieves data from the clipboard, converts it to JSON format, and prepares it to be sent to the server.

[1598] Step 3:

[1599] The server receives the dataset sent from the device and converts it into a specific data format using a generative model.

[1600] Input: Dataset submitted in JSON format.

[1601] Output: A dataset converted into a specific data format (e.g., SQL format).

[1602] Specific operation: The server receives the dataset and converts it into SQL format using a generative model (e.g., GPT-3).

[1603] Step 4:

[1604] The server returns the transformed data set to the user terminal.

[1605] Input: A dataset converted into a specific data format.

[1606] Output: The dataset that is sent back to the user's device.

[1607] Specific operation: The converted dataset is sent to the user terminal as a response to the HTTP request.

[1608] Step 5:

[1609] The terminal provides the received converted data to the user and displays it.

[1610] Input: A dataset returned from the server in a specific data format.

[1611] Output: The data format, such as an SQL query, that is made available to the user.

[1612] Specific operation: The converted dataset is displayed on the terminal, and the user pastes it into the database for use.

[1613] Step 6:

[1614] Requires the user to type or copy and paste program code (e.g., SQL code) and insert line breaks at the appropriate places.

[1615] Input: Program code typed or pasted by the user.

[1616] Output: The program code sent from the terminal to the server.

[1617] What it does: Copies the code the user types or pastes to the clipboard and prepares it to be sent to the server.

[1618] Step 7:

[1619] The server analyzes the received program code and uses a generative model to insert line breaks at the appropriate positions.

[1620] Input: The program code sent from the terminal.

[1621] Output: Program code with line breaks inserted.

[1622] Specific operation: The code received by the server is analyzed using a generative model, and line breaks are automatically determined and inserted to improve readability.

[1623] Step 8:

[1624] The server returns the program code with line breaks inserted to the terminal.

[1625] Input: Program code with line breaks inserted.

[1626] Output: Program code returned to the terminal with line breaks.

[1627] Specific operation: The program code after the line break is sent to the terminal as a response to the HTTP request.

[1628] Step 9:

[1629] The terminal displays the program code after the line break to the user.

[1630] Input: The program code returned from the server after the line break.

[1631] Output: Readable program code available to the user.

[1632] Specific operation: The program code after line breaks is displayed on the terminal so that the user can use it as is.

[1633] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1634] This invention relates to a system that not only improves the efficiency of data processing work using computer software, but also provides a feedback adjustment function based on user emotion recognition. Specifically, it is composed of a data processing device that converts data formats, inserts line breaks into program code, and recognizes emotions using an emotion engine.

[1635] composition

[1636] This system consists of a user device, a server, a generative model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[1637] What the program does

[1638] Data format conversion

[1639] 1. Data selection and acquisition

[1640] The user selects the required data in a calculation software (e.g. Excel). After selection, the data is copied.

[1641] The terminal retrieves the selected data from the clipboard and prepares it for transmission to the server.

[1642] 2. Data transmission and conversion

[1643] The device sends the acquired data to the server via an HTTP request. The data is packaged in JSON format or similar.

[1644] The server parses the received data and converts it into a specific SQL format using a generative model.

[1645] 3. Return and display of converted data

[1646] The server sends the converted SQL data to the terminal as an HTTP response.

[1647] The terminal displays the received SQL data to the user.

[1648] Inserting a line break

[1649] 1. Enter and submit SQL code

[1650] The user types or pastes SQL code into the terminal.

[1651] The terminal sends the entered code to the server.

[1652] 2. Line break handling

[1653] The server parses the received SQL code and uses the generative model to insert line breaks at the appropriate places.

[1654] 3. Returning and displaying the code after the line break

[1655] The server sends the SQL code with line breaks inserted to the terminal.

[1656] The terminal re-displays the received SQL code after line breaks in the input field.

[1657] Emotion recognition by emotion engine

[1658] 1. Collecting Emotional Data

[1659] The device uses a microphone and camera to collect the user's voice and facial expressions.

[1660] 2. Emotion Recognition Analysis

[1661] The server uses an emotion engine to recognize the user's emotional state from the collected voice and facial expression data, for example classifying emotions as positive, negative, neutral, etc.

[1662] 3. Adjusting feedback

[1663] The server adjusts the feedback of the data processing based on the recognized emotion, for example, providing more detailed explanations or guidance if the user is showing negative emotions.

[1664] 4. Providing Feedback

[1665] The terminal provides the user with the feedback received from the server, thereby improving the user's work efficiency and satisfaction.

[1666] Specific examples

[1667] Data format conversion example

[1668] A user selects and copies the columns "Year," "Sales," and "Expenses" in Excel.

[1669] The terminal sends this data to the server, which converts it into SQL format: "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1670] The converted SQL data is sent back to the terminal, and the user pastes the specified SQL format into the database to use it.

[1671] Example of inserting a line break

[1672] A user enters a long piece of SQL code such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1673] The terminal sends this code to the server, which then inserts line breaks in the appropriate places and converts it to something like "SELECT \nFROM Sales\nWHERE Year = '2023'\nAND Revenue > 5000;".

[1674] The code after line breaks is sent back to the terminal, where the user can confirm the improved readability and use the code.

[1675] Example of feedback adjustment using emotion recognition

[1676] While the user is using the system, the terminal collects the user's voice and facial expressions.

[1677] The server uses an emotion engine to recognize that the user is confused.

[1678] The server generates a more detailed help message and sends it to the terminal.

[1679] The terminal displays a help message to the user, and the user receives assistance in resolving the problem.

[1680] The configuration and processing of the present invention allow users to quickly and efficiently convert data into specific data formats and improve the readability of program code, and also provide a better user experience through feedback adjustment based on emotion recognition.

[1681] The processing flow will be explained below.

[1682] Data format conversion

[1683] Step 1:

[1684] The user selects the required data in calculation software (e.g. Excel). For example, the user selects a range of columns such as "Fiscal Year," "Sales," and "Expenses."

[1685] Step 2:

[1686] Copies the data selected by the user. The selected data is transferred to the clipboard.

[1687] Step 3:

[1688] The device retrieves the selected data from the clipboard using the clipboard monitor function.

[1689] Step 4:

[1690] The data acquired by the device is sent to the server via an HTTP request. The data is packaged in JSON format or similar.

[1691] Step 5:

[1692] The server analyzes the data received. Specifically, it parses the JSON format data and extracts the column names and data for each row.

[1693] Step 6:

[1694] The server uses the generative model to convert the data into SQL format, for example generating the SQL statement "INSERT INTO Sales (Year, Revenue, Expense) VALUES ...".

[1695] Step 7:

[1696] The server sends the converted SQL data to the terminal as an HTTP response.

[1697] Step 8:

[1698] The terminal displays the received SQL data to the user in a dedicated window or message box.

[1699] Step 9:

[1700] The user checks the displayed SQL data and pastes it into a database for use.

[1701] Inserting a line break

[1702] Step 1:

[1703] A user types or copies and pastes a long piece of SQL code into an input field on a terminal, such as "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000".

[1704] Step 2:

[1705] The terminal retrieves the SQL code from the input field. This operation is triggered by an input event or similar.

[1706] Step 3:

[1707] The terminal sends the acquired SQL code to the server via an HTTP request. The code is packaged in text format.

[1708] Step 4:

[1709] The server analyzes the received SQL code, specifically by performing text analysis and generating a syntax tree.

[1710] Step 5:

[1711] The server uses the generated model to insert appropriate line breaks into the SQL code, for example, before and after specified keywords.

[1712] Step 6:

[1713] The server sends the SQL code with line breaks inserted to the terminal as an HTTP response.

[1714] Step 7:

[1715] The SQL code received by the terminal is redisplayed in the input field after line breaks.

[1716] Step 8:

[1717] The user can review the displayed SQL code and edit or use it as needed.

[1718] Emotion recognition and feedback adjustment by emotion engine

[1719] Step 1:

[1720] The device collects the user's voice and facial expression data in real time using a microphone and camera, for example, capturing the user's face with a webcam.

[1721] Step 2:

[1722] The device will transmit the collected voice and facial expression data to a server, using appropriate data security techniques.

[1723] Step 3:

[1724] The server uses an emotion engine to analyze the user's voice and facial expression data, for example, by evaluating voice tone and facial muscle movements to identify emotions.

[1725] Step 4:

[1726] The server adjusts the feedback of the data processing based on the perceived emotion, for example generating a more detailed help message if it determines that the user is confused.

[1727] Step 5:

[1728] Sends server-generated feedback to the device, which may include additional explanations or guidance.

[1729] Step 6:

[1730] The device displays the received feedback to the user, allowing the user to work more efficiently.

[1731] The above are the specific processing steps for data format conversion, line break insertion, and feedback adjustment through emotion recognition in this system.

[1732] Example 2

[1733] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1734] In conventional data processing systems, the format conversion of user-specified data sets and the formatting of program code are often performed manually, which reduces efficiency and leads to errors.In addition, one-sided feedback is provided without considering the user's emotional state, which leads to a lack of user satisfaction.

[1735] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1736] In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative AI model to convert the dataset into a specific data format, means for acquiring data from a clipboard, means for providing the converted dataset to the user, means for collecting voice and facial expressions to recognize the emotional state of the user, and means for recognizing emotions from the collected data using an emotion engine and adjusting feedback based on the recognized emotions. This allows for efficient automatic data conversion and program code formatting, and further enables appropriate feedback to be provided based on the user's emotional state.

[1737] A "dataset" is a set of data that a user specifies they want to process.

[1738] A "generative AI model" is an algorithm that uses machine learning techniques to generate specific outputs based on large amounts of data.

[1739] The "clipboard" is a memory area that temporarily stores data within a terminal and enables data transfer between other applications.

[1740] "Emotion engine" is a general term for algorithms and software that analyze voice and facial expression data to identify a user's emotional state.

[1741] "Feedback" refers to the reactions and information provided by a system in response to a user's actions and emotions.

[1742] A "user" is a person or entity that is responsible for operating the system and providing data and instructions.

[1743] The present invention is a system that utilizes computer software to provide efficient data processing and feedback adjustment based on user emotion recognition. Specifically, it constitutes a data processing device that includes data format conversion, line break insertion in program code, and emotion recognition using an emotion engine.

[1744] composition

[1745] This system consists of a user device, a server, a generative AI model, and an emotion engine. The user device functions as an interface for data input, reception, and transmission, while the server is the center of data processing and conversion. The generative AI model is located on the server and processes data using specific algorithms and machine learning techniques. The emotion engine recognizes the user's emotional state and reflects it as feedback in the data processing results.

[1746] Data format conversion

[1747] In data format conversion, the user selects the required data in calculation software (e.g., Excel) and copies it to the clipboard. The terminal retrieves the data from the clipboard and sends it to the server. The server analyzes the received data and converts it into SQL format (e.g., INSERT INTO Sales (Year, Revenue, Expense) VALUES ...) using a generative AI model. The converted SQL data is sent back from the server to the terminal and displayed to the user. An example of an extracted prompt sentence is as follows:

[1748] Example prompt sentence:

[1749] Convert data containing columns such as "Year", "Revenue", and "Expense" into SQL format (e.g. INSERT INTO Sales (Year, Revenue, Expense) VALUES ...).

[1750] Inserting a line break

[1751] To insert a line break into SQL code, the user enters or pastes the SQL code on the terminal. The terminal sends the entered code to the server, which analyzes the received SQL code and uses a generative AI model to insert line breaks in the appropriate positions. The SQL code with line breaks is then sent back to the terminal and redisplayed to the user. An example of an extracted prompt sentence is as follows:

[1752] Example prompt sentence:

[1753] Insert line breaks at the appropriate positions in the following SQL code: Code: SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000;

[1754] Emotion recognition by emotion engine

[1755] In emotion recognition using an emotion engine, the device uses a microphone and camera to collect the user's voice and facial expression data. The collected data is sent to the server in real time, where it is analyzed and used by the emotion engine to recognize the user's emotional state. For example, emotions are classified as positive, negative, or neutral. Based on the recognized emotional state, the server adjusts the feedback and generates detailed guidance and explanations. The generated feedback is sent to the device and displayed to the user. An example of an extracted prompt sentence is as follows:

[1756] Example prompt sentence:

[1757] Analyze voice and facial expression data to classify the user's emotional state (positive, negative, neutral, etc.). If it is negative, generate a detailed help message to provide feedback.

[1758] This allows users to quickly and efficiently convert data into specific formats, improve the readability of program code, and receive appropriate feedback through emotion recognition.

[1759] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1760] Data format conversion

[1761] Step 1:

[1762] The user opens a calculation software (e.g. Excel) and selects the columns "Year", "Sales", and "Expenses" to copy. The input of this step is the data selected by the user, and the output is the data copied to the clipboard.

[1763] Step 2:

[1764] The terminal gets data from the clipboard and sends it to the server. The input of this step is the data stored in the clipboard, and the output is the JSON formatted data that is sent to the server.

[1765] Step 3:

[1766] The server parses the received data and converts it into a specific SQL format using a generative AI model. The input of this step is JSON formatted data and the output is SQL formatted data.

[1767] Step 4:

[1768] The server sends the converted SQL data to the terminal as an HTTP response. The input of this step is data in SQL format, and the output is the data sent to the terminal as an HTTP response.

[1769] Step 5:

[1770] The terminal formats the received SQL data and displays it to the user. The input to this step is the SQL data received from the server, and the output is the formatted SQL data that is displayed to the user.

[1771] Inserting a line break

[1772] Step 1:

[1773] The user types SQL code into their terminal (e.g., "SELECT FROM Sales WHERE Year = '2023' AND Revenue > 5000") or pastes it from the clipboard. The input for this step is the SQL code typed or pasted by the user, and the output is the SQL code displayed on the terminal.

[1774] Step 2:

[1775] The terminal sends the entered or pasted SQL code to the server. The input of this step is the SQL code entered or pasted by the user, and the output is the SQL code sent to the server.

[1776] Step 3:

[1777] The server parses the received SQL code and uses a generative AI model to insert line breaks at the appropriate positions. The input to this step is the received SQL code, and the output is the SQL code with line breaks inserted.

[1778] Step 4:

[1779] The server sends the SQL code with line breaks inserted to the terminal. The input of this step is the SQL code with line breaks inserted, and the output is the SQL code with line breaks sent to the terminal.

[1780] Step 5:

[1781] The terminal re-displays the received SQL code with line breaks to the user. The input of this step is the SQL code received from the server with line breaks, and the output is the SQL code with line breaks re-displayed to the user.

[1782] Emotion recognition by emotion engine

[1783] Step 1:

[1784] The device uses a microphone and a camera to collect the user's voice and facial expressions. The input of this step is the user's voice and facial expression data, and the output is the collected real-time emotion data.

[1785] Step 2:

[1786] The terminal transmits the collected emotion data to the server in real time. The input of this step is the collected emotion data, and the output is the emotion data transmitted to the server.

[1787] Step 3:

[1788] The server uses an emotion engine to recognize the user's emotional state from the transmitted voice and facial expression data. The input of this step is the collected voice and facial expression data, and the output is the recognized user's emotional state (e.g., positive, negative, neutral).

[1789] Step 4:

[1790] The server adjusts the feedback based on the recognized emotional state and generates detailed guides and explanations. The input of this step is the recognized emotional state and the output is the adjusted feedback data.

[1791] Step 5:

[1792] The server sends the generated feedback data to the terminal. The input of this step is the generated feedback data, and the output is the feedback data sent to the terminal.

[1793] Step 6:

[1794] The terminal displays the feedback received from the server to the user. The input of this step is the feedback data received from the server, and the output is the feedback displayed to the user.

[1795] (Application example 2)

[1796] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1797] Conventional data processing systems have limitations in converting user-specified data sets and improving the readability of program code. It is also difficult for systems to recognize users' emotional states and adjust feedback in real time, which is particularly important for improving the quality of customer service in brick-and-mortar stores. The present invention aims to solve these problems and significantly improve the user experience.

[1798] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a dataset specified by a user, means for using a generative model to convert the dataset into a specific data format, means for providing the converted dataset to the user, means for using an emotion engine to recognize the emotional state of the user, and means for adjusting feedback based on the recognized emotional state. This enables the system to quickly and efficiently convert into a data format, improve the readability of program code, and adjust feedback based on emotion recognition.

[1799] A "dataset" refers to a series of data items or information specified by a user.

[1800] A "generative model" is a model that uses machine learning algorithms to transform or reorganize data into a specific format.

[1801] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state from input data such as voice and facial expressions and recognizes that state.

[1802] "Feedback" refers to the responses or instructions provided by a system based on a user's input or emotional state.

[1803] "Server" refers to a computer system that acts as a data processing and transformation center.

[1804] The system for implementing this invention consists of four main components: a server, a user terminal, a generative model, and an emotion engine. Below, we will provide an overview of the functions and processing of each of these components.

[1805] System Configuration

[1806] The system has the ability to receive user-specified data sets and convert them into specific data formats, as well as to insert line breaks at appropriate positions in user-entered program code and to recognize the user's emotional state and adjust feedback accordingly.

[1807] User terminal

[1808] The user terminal can use smart glasses and function as an interface for data input, reception, and transmission. This allows the user's voice and facial expressions to be collected in real time and transmitted to the server. The server also displays feedback to the user.

[1809] server

[1810] The server acts as a data processing and transformation center, using a generative model to transform the data and returning the converted dataset to the user in a specific data format. It also uses an emotion engine to recognize the user's emotional state and generate feedback based on that.

[1811] Generative Model

[1812] Generative models are used to transform data into specific formats or to insert line breaks into program code. They use machine learning algorithms to transform input data efficiently and accurately.

[1813] Emotion Engine

[1814] The emotion engine analyzes and recognizes the user's emotional state from input data such as voice and facial expressions, and if the emotional state is negative, it provides the user with detailed guidance and support messages.

[1815] System Processing Overview

[1816] Data format conversion

[1817] 1. Receiving a user-specified dataset

[1818] The user inputs a dataset using voice commands through the smart glasses. The device receives the data and prepares it for transmission to the server.

[1819] 2. Data format conversion by the server

[1820] The server uses the generative model to transform the received dataset into a specific format, for example, into an SQL query or a specific database format.

[1821] Emotion Recognition and Feedback

[1822] 1. Recognizing the user's emotional state

[1823] The smart glasses use a built-in camera and microphone to collect the user's voice and facial expressions, and the server uses an emotion engine to recognize the user's emotional state from these data.

[1824] 2. Adjusting feedback

[1825] The server adjusts the feedback based on the perceived emotional state, providing detailed help messages and direct support if negative emotions are detected.

[1826] Specific examples

[1827] For example, when a user voice-inputs a customer's order details through the smart glasses, the data set is sent from the smart glasses to the server and converted into an SQL query format. If the user's facial expression is confused, the server's emotion engine will recognize it and display a detailed guide message on the smart glasses.

[1828] Prompt Sentence Examples

[1829] Get customer order details and convert them into SQL format. Also, recognize the customer's emotional state and provide a support message if necessary.

[1830] This system is expected to improve customer service efficiency in physical stores and increase customer satisfaction.

[1831] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1832] Step 1:

[1833] The user specifies the customer's order by voice input via the smart glasses. The microphone in the smart glasses captures the voice data and converts it into text data using voice recognition software (e.g., Google Cloud Speech-to-Text). This text data is then input into the user's device.

[1834] Step 2:

[1835] The user device packages the acquired text data in JSON format and sends it to the server using an HTTP request. The server receives the request and parses the data using a JSON parser (e.g., json.loads).

[1836] Step 3:

[1837] The server inputs the analyzed data into a generative model (e.g., GPT-3) and performs data processing to convert it into a specific data format (e.g., SQL query). The server receives the SQL query output by the generative model.

[1838] Step 4:

[1839] The server sends the generated SQL query as an HTTP response to the user terminal, which receives the HTTP response and displays the acquired SQL query on the display of the smart glasses.

[1840] Step 5:

[1841] The smart glasses' camera and microphone are used to collect the customer's voice and facial expressions in real time, and this data is then input into the device, which then sends it to the server as an HTTP request.

[1842] Step 6:

[1843] The server inputs the received voice and facial expression data into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze and recognize the customer's emotional state. The emotion engine outputs the emotion recognition results to the server.

[1844] Step 7:

[1845] The server generates feedback based on the recognized emotional state. For example, if the customer is confused, it generates a detailed guide message. The server sends this feedback to the terminal as an HTTP response.

[1846] Step 8:

[1847] The user terminal displays the feedback received from the server on the display of the smart glasses, for example, a message such as "What's the matter? We'll show you around."

[1848] This series of processes is expected to significantly improve the quality and efficiency of customer service.

[1849] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1850] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1851] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1852] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1853] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1854] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1855] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1856] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1857] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1858] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1859] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1860] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1861] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1862] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1863] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1864] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1865] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1866] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1867] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1868] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1869] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1870] The following is further disclosed regarding the above embodiment.

[1871] (Claim 1)

[1872] 1. A system including a data processing device,

[1873] The data processing device

[1874] means for receiving a user-specified data set;

[1875] means for using a generative model to convert the dataset into a particular data format;

[1876] means for providing the transformed data set to a user;

[1877] A system including:

[1878] (Claim 2)

[1879] The data processing device

[1880] 10. The system of claim 1, further comprising means for using a generative model to insert line breaks at appropriate positions in input program code.

[1881] (Claim 3)

[1882] The data processing device

[1883] 10. The system of claim 1, further comprising means for obtaining data from a clipboard.

[1884] "Example 1"

[1885] (Claim 1)

[1886] means for receiving a user-specified data set;

[1887] means for using a generative AI model to convert the dataset into a particular data format;

[1888] means for providing the transformed data set to a user;

[1889] a means for retrieving data from the clipboard;

[1890] means for transmitting the acquired data to a server;

[1891] A means for the server to parse the data it receives and convert it into the appropriate SQL format;

[1892] means for returning the converted SQL format data to a user terminal;

[1893] A means for displaying the received SQL data by the user terminal;

[1894] A system including:

[1895] (Claim 2)

[1896] 10. The system of claim 1, further comprising means for using a generative AI model to insert line breaks at appropriate locations for input program code.

[1897] (Claim 3)

[1898] 2. The system according to claim 1, further comprising means for inserting appropriate line breaks into the program code received by the server and returning the code after the line breaks to the user terminal.

[1899] "Application Example 1"

[1900] (Claim 1)

[1901] means for receiving a data set designated by the user terminal;

[1902] means for using a generative model to convert the dataset into a particular data format;

[1903] A means for using a generative model to insert line breaks at appropriate positions in input program code;

[1904] means for providing the converted data set and the program code after line breaks to a user;

[1905] a means for the system to centrally manage data related to factory automation;

[1906] means for automatically converting said data into a required data format;

[1907] means for inserting line breaks into said SQL code to improve readability;

[1908] means for transmitting and managing the program code to a server;

[1909] A system including:

[1910] (Claim 2)

[1911] 10. The system of claim 1, wherein the user terminal further comprises means for obtaining data from the clipboard.

[1912] (Claim 3)

[1913] The system according to claim 1, characterized in that the system is installed on a robot for factory automation, and centrally manages and efficiently processes work data.

[1914] "Example 2: Combining Emotion Engines"

[1915] (Claim 1)

[1916] means for receiving a user-specified data set;

[1917] means for using a generative AI model to convert the dataset into a particular data format;

[1918] a means for retrieving data from the clipboard;

[1919] means for providing the transformed data set to a user;

[1920] means for collecting voice and facial expressions to recognize the emotional state of the user;

[1921] means for recognizing emotions from the collected data using an emotion engine and adjusting feedback based on the recognized emotions;

[1922] A system including:

[1923] (Claim 2)

[1924] 10. The system of claim 1, further comprising means for using a generative AI model to insert line breaks at appropriate locations for input program code.

[1925] (Claim 3)

[1926] 10. The system of claim 1, further comprising means for providing the user with program code with line breaks inserted.

[1927] "Application example 2 when combining emotion engines"

[1928] (Claim 1)

[1929] means for receiving a user-specified data set;

[1930] means for using a generative model to convert the dataset into a particular data format;

[1931] means for providing the transformed data set to a user;

[1932] means for using an emotion engine to recognize the emotional state of a user;

[1933] means for adjusting feedback based on the recognized emotional state;

[1934] A system including:

[1935] (Claim 2)

[1936] 10. The system of claim 1, further comprising means for using a generative model to insert line breaks at appropriate positions in input program code.

[1937] (Claim 3)

[1938] 10. The system of claim 1, further comprising means for obtaining data from a clipboard. [Explanation of symbols]

[1939] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. 1. A system including a data processing device, The data processing device means for receiving a user-specified data set; means for using a generative model to convert the dataset into a particular data format; means for providing the transformed data set to a user; A system including:

2. The data processing device 10. The system of claim 1, further comprising means for using a generative model to insert line breaks at appropriate positions in input program code.

3. The data processing device 10. The system of claim 1, further comprising means for obtaining data from a clipboard.

Images