System

Abstract

A system is provided.SOLUTION: A system including a means for inputting information related to a difficulty of a disabled person, a means for transmitting the input information to a server, a means for receiving the input information and searching for an optimal support method using a generated AI model by the server, a means for transmitting a support method and a practical example generated by the server using the generated AI model to a terminal, and a means for displaying the support method and the practical example transmitted by the terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the past, providing appropriate support to children with disabilities relied heavily on the individual experience and abilities of teachers, making it difficult to quickly find standardized, effective support methods. Furthermore, while the number of children requiring support is increasing due to the declining birthrate, the teacher shortage is becoming more serious. In such a situation, it is difficult to provide support that is appropriate for each individual, and there is a risk that the quality of support will decline. This could have a significant impact on children's growth and development. The purpose of this project is to solve this problem. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes a means for inputting information about the difficulties faced by individuals with disabilities, a means for transmitting the input information to a server, a means for the server to receive the input information and search for optimal support methods using a generative AI model, a means for the server to transmit support methods and practical examples generated using the generative AI model to a terminal, and a means for the terminal to display the transmitted support methods and practical examples. This enables teachers to provide appropriate support to children with disabilities quickly and effectively. Furthermore, the use of preprocessing means and structured data formats enables efficient and secure data exchange.

[0006] A "disability" is a condition resulting from a disease or accident that is present or occurs after birth, resulting in some significant limitation of physical or mental functions.

[0007] "Difficulties" refer to the specific problems and difficulties that individuals with disabilities face in their daily lives and learning.

[0008] "Input means" refers to the interface or device that a user uses to input information.

[0009] A "server" is a computer system that can receive, process, and send data over a network.

[0010] A "generative AI model" is a system based on machine learning algorithms that automatically generates appropriate assistance methods based on input data.

[0011] "Searching means" refers to the process by which the server identifies the most appropriate support method based on the input data.

[0012] "Practical examples" refer to data showing specific methods and results used to solve similar problems in the past.

[0013] A "terminal" is a device that allows a user to input and display information, and includes smartphones, personal computers, and the like.

[0014] "Preprocessing" refers to the process of converting raw data into a form suitable for a generative AI model.

[0015] A "structured data format" is a data format in which data is organized in an orderly manner according to a certain format, and includes JSON and XML.

[0016] "Secure" refers to the safe transmission and reception of data, and is a state in which data is protected from unauthorized access by third parties.

[0017] "Efficient" refers to achieving a goal while minimizing resources (time, computing power, etc.). [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is a system that supports individual support for children with disabilities. In particular, the system allows users (teachers) to input information about disabilities and difficulties, and uses a generative AI model to search for optimal support methods and practical examples, displaying the results on a terminal to provide fast and effective support.

[0040] System Configuration

[0041] This system mainly consists of the following components:

[0042] 1. User terminal: A device used by the teacher that provides an interface for entering information and displaying results.

[0043] 2. Server: Receives information from the user device, executes the generative AI model, and generates appropriate support methods and practical examples.

[0044] 3. Generative AI model: A machine learning algorithm that searches for and generates optimal support methods based on information about disabilities and difficulties.

[0045] Program processing

[0046] The program process for this system is as follows:

[0047] User Input

[0048] The user (teacher) uses the device interface to input information about the child's disability or difficulties, such as information about autism spectrum disorder or difficulties with social interaction.

[0049] Sending data

[0050] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using a secure protocol (HTTPS). At this stage, the consistency and integrity of the data is ensured.

[0051] Server reception and analysis

[0052] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and feeds it into a generative AI model. This preprocessing transforms the data into a format that is easier to analyze.

[0053] Running generative AI

[0054] The server's generative AI model searches for the optimal support method based on the input data and generates related practical examples, such as "communication support using picture cards."

[0055] Processing and sending results

[0056] The server then reorganizes the generated support methods and practical examples in a structured data format and sends it to the terminal. At this stage, the data is organized in a form that can be used immediately by the user.

[0057] Displaying the results

[0058] The device interprets the received data and displays it in a format that is easy for the user to understand. For example, it may visually present something like "Communication support methods for autism spectrum disorders" on the screen.

[0059] Specific examples

[0060] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," this information is sent to the server. The server uses generative AI to generate a result that "communication practice using picture cards" is effective, and sends this result along with appropriate examples to the device. As a result, the user can quickly obtain specific support methods and apply them in actual support.

[0061] As described above, the present invention specifically shows an embodiment for quickly and effectively providing an appropriate support method for children with disabilities.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[0065] Step 2:

[0066] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent.

[0067] Step 3:

[0068] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[0069] Step 4:

[0070] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[0071] Step 5:

[0072] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[0073] Step 6:

[0074] The server inputs the preprocessed data into a generative AI model, which generates optimal support methods and practical examples based on the input data. For example, it outputs a specific support method such as "communication practice using picture cards."

[0075] Step 7:

[0076] The server organizes the output from the generative AI model into a data package, which includes details of assistance methods and practical examples.

[0077] Step 8:

[0078] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[0079] Step 9:

[0080] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0081] Step 10:

[0082] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] It is difficult to find a method for providing individualized support to children with disabilities quickly and effectively using conventional methods. In particular, they lack the ability to properly analyze input information and generate specific and practical support methods. There is also a need to display the generated information in a format that is easy for users to understand. The present invention aims to provide a system that solves these problems.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes means for receiving input information and performing preprocessing such as text normalization and tokenization, means for generating optimal support methods and practical examples from the preprocessed information using a generative AI model, and means for transmitting the generated support methods and practical examples to the terminal in structured data format, thereby making it possible to quickly and effectively analyze the input information and provide specific and practical support methods to the user.

[0088] "Disability and difficulty information" is data that details the specific circumstances, problems, and difficulties of individuals with disabilities.

[0089] "Structured data" is a data format that is organized based on a specific format or standard, and is usually expressed in formats such as JSON or XML.

[0090] A "secure communication protocol" is a communication technology that prevents data exchange from being eavesdropped on or tampered with by third parties, and includes HTTPS.

[0091] "Preprocessing" refers to processes such as text normalization and tokenization to make data easier for generative AI models to parse.

[0092] "Normalization" refers to the process of organizing text data by removing unnecessary symbols and spaces.

[0093] "Tokenization" refers to the process of dividing text data into units such as words and phrases.

[0094] A "generative AI model" is a model that uses machine learning algorithms to generate optimal support methods and practical examples based on input data.

[0095] "Assistive techniques" refers to specific aids or approaches provided to individuals with disabilities.

[0096] "Practical cases" are data that describe specific successful cases and methods of support that have been provided in the past.

[0097] A "structured data format" is a form of data that is systematically organized in a specific format, commonly including JSON and XML.

[0098] A "terminal" is a device, such as a computer, tablet, or smartphone, that a user uses to enter information and view results.

[0099] The "format that is easy for users to understand" is a format in which the generated support methods and practical examples are displayed in a visually easy-to-understand manner so that the user can easily understand them.

[0100] This invention is a system that provides individualized support to children with disabilities quickly and effectively. This system mainly consists of a user terminal, a server, and a generative AI model.

[0101] User terminal

[0102] User terminals are devices used by teachers and supporters, and provide an interface for inputting information about children's disabilities and difficulties. Specifically, PCs, tablets, smartphones, etc. are used. The terminals are equipped with a dedicated form where users can enter various information, and the information entered through this form is converted into structured data (e.g., JSON format). This structured data is sent to the server using a secure communication protocol such as HTTPS.

[0103] server

[0104] The server receives the information sent from the user device and performs preprocessing on the data. Specific preprocessing includes normalizing and tokenizing the text. For example, it removes unnecessary symbols and spaces from the text and checks the integrity of the data. This preprocessing prepares the data in a format that is easy for the generative AI model to analyze.

[0105] The server uses a generative AI model to generate optimal support methods and practical examples from the preprocessed data. The generated support methods and practical examples are then organized again in a structured data format and sent to the user's device. At this time, the generated information is prepared to be displayed in a visually easy-to-understand format.

[0106] Generative AI Models

[0107] The generative AI model uses machine learning algorithms to generate optimal support methods and practical examples based on input data. This server-based model analyzes detailed information about disabilities and problems, and suggests optimal support methods based on past support data.

[0108] Specific examples

[0109] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction" on a device, this information is converted into JSON format and sent to the server using the secure HTTPS protocol. The server receives this information and normalizes and tokenizes the text. The generative AI model then analyzes this data and generates a "communication support method using picture cards" and specific practical examples. This generated information is then sent back to the device in structured data format and displayed in a format that is easy for the user to understand.

[0110] Prompt Sentence Examples

[0111] Examples of prompts that users can input to a generative AI model include:

[0112] "When a user inputs the information 'autism spectrum disorder' and 'difficulty in social interaction,' please generate the optimal assistance method."

[0113] Thus, the present invention specifically illustrates an embodiment for providing individualized support to children with disabilities quickly and effectively.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] The user (teacher) uses the terminal interface to input information about the child's disability or difficulties. Specifically, the user enters information such as "autism spectrum disorder" or "difficulty in social interaction" into an input form. This input is provided by the user in the form of text data.

[0117] Input: Text data about a child's disability or difficulties (e.g., "autism spectrum disorder" or "difficulty with social interaction")

[0118] Output: Structured data (JSON format)

[0119] Step 2:

[0120] The terminal converts the information entered by the user into structured data (JSON format), which organizes the data in a consistent format, and then transmits the converted data to the server using a secure communication protocol (HTTPS).

[0121] Input: Text data (information entered by the user)

[0122] Output: Structured data (JSON format), submitted data

[0123] Step 3:

[0124] The server receives the structured data sent from the terminal. The received data is first preprocessed, such as text normalization and tokenization, to remove unnecessary symbols and whitespace to ensure data consistency and integrity.

[0125] Input: Structured data (JSON format)

[0126] Output: Preprocessed data (text normalized, tokenized data)

[0127] Step 4:

[0128] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed data. The generative AI model uses machine learning algorithms to analyze the data and generate optimal solutions by referring to past case data. In this process, new support methods and practical examples are created based on the data entered by the user.

[0129] Input: Preprocessed data (text normalized, tokenized data)

[0130] Output: Support methods and practical examples

[0131] Step 5:

[0132] The server then organizes the generated support methods and practical examples in a structured data format (JSON format) and sends it to the user's device. At this stage, the data is formatted so that it can be displayed visually in an easy-to-understand manner.

[0133] Input: Support methods and practical examples

[0134] Output: Structured data (JSON format), submitted structured data

[0135] Step 6:

[0136] The device interprets the structured data received from the server and displays it in a format that is easy for the user to understand. Specifically, it visually presents specific support methods and practical examples along with titles such as "Communication Support Methods for Autism Spectrum Disorder."

[0137] Input: Structured data (JSON format), data received from the server

[0138] Output: A visually easy-to-understand display for the user

[0139] At each step, data processing and calculations are performed based on the input and output to ensure the smooth functioning of the entire system, and the system is designed to enable users to obtain assistance methods quickly and effectively.

[0140] (Application example 1)

[0141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] In addition to providing appropriate assistance methods for individuals with disabilities, there is a need for a system that quickly provides efficient work procedures and problem-solving methods to factory workers. However, because these assistance methods and work procedures vary depending on individual circumstances, a one-size-fits-all solution is insufficient. Furthermore, delays in providing real-time assistance methods and work procedures can lead to reduced productivity and work efficiency.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0144] In this invention, the server includes: means for inputting information about a problem faced by an individual with a disability; means for transmitting the input information to the server; means for the server receiving the input information and searching for an optimal assistance method using a generative AI model; means for the server transmitting the assistance method generated using the generative AI model and practical examples to a terminal; means for the terminal displaying the transmitted assistance method and practical examples; means for the terminal inputting a factory work efficiency improvement method or problem-solving method and generating an optimal work procedure using a generative AI model; means for the server transmitting the generated work procedure and related examples to the terminal; and means for the terminal displaying the transmitted work procedure and related examples. This makes it possible to provide not only appropriate and personalized assistance methods for individuals with disabilities, but also efficient work procedures and problem-solving methods in real time to factory workers.

[0145] "Individual with a disability" means a person who has a physical or mental impairment.

[0146] "Information about problems" refers to data about specific challenges and problems that individuals face in their daily lives and at work.

[0147] "Server" refers to a computer system that receives, processes, and transmits information over a network.

[0148] A "generative AI model" refers to an artificial intelligence system that uses machine learning algorithms to automatically generate optimal solutions and procedures.

[0149] "Optimal support methods" refer to the most effective and appropriate responses and support methods for an individual's problems.

[0150] "Practical examples" refer to examples of support methods or solutions that have been shown to be effective based on specific situations or cases.

[0151] A "terminal" is a device that allows users to input and output data, and in this case refers to tablets and smart glasses used by factory workers.

[0152] "Improving factory work efficiency" refers to improving productivity by optimizing work procedures and the work environment in a factory.

[0153] A "problem-solving method" refers to a means or procedure for solving a specific problem or obstacle.

[0154] "Work procedures" refer to the specific steps and methods for performing work or operations within a factory.

[0155] "Relevant examples" refer to specific examples or case studies of successful implementations of similar situations or challenges.

[0156] This invention is a system that quickly provides appropriate support methods and efficiency procedures for individuals and factory workers with disabilities. It mainly consists of the following specific components:

[0157] 1. User Device:

[0158] This is a device for inputting information about the difficulties faced by individuals with disabilities and the efficiency of factory work. This includes devices such as tablets and smart glasses. For example, workers can input information about "pick-and-place operations."

[0159] 2. Server:

[0160] This is a computer system that receives and preprocesses information sent from the user's device. This server contains a generative AI model and automatically generates optimal support methods and work procedures based on the input information. Specifically, cloud-based servers such as AWS (registered trademark) EC2 and Google (registered trademark) Cloud Platform are often used.

[0161] 3. Generative AI Model:

[0162] This is an AI system that uses machine learning algorithms to generate optimal solutions based on input information, such as OpenAI's GPT series. This AI model provides personalized assistance methods for individuals with disabilities and optimal work procedures for factory workers.

[0163] Example

[0164] Improving factory work efficiency

[0165] Users input information about issues with "pick-and-place operations" and "frequent machine stoppages" via a tablet. Based on this, the server uses a generative AI model to generate optimal work procedures, such as "recalibrating sensors" and "introducing a high-speed vision system." The generated procedures and related cases are sent to the terminal, allowing workers to take immediate action.

[0166] For example, enter the following prompt:

[0167] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[0168] Hardware and software used

[0169] Hardware:

[0170] Smart glasses (e.g., Microsoft® HoloLens®)

[0171] Tablets (e.g. iPad(R) Pro)

[0172] Factory robots (e.g. ABB IRB series, KUKA KR series)

[0173] software:

[0174] Cloud servers (e.g. AWS EC2, Google Cloud Platform)

[0175] Machine learning API (e.g. OpenAI GPT series)

[0176] Data processing and calculation

[0177] The server first preprocesses the information received from the user device. This preprocessing includes text normalization and tokenization. The generative AI model then generates optimal assistance methods and work procedures based on the preprocessed data. This generated data is then restructured and sent to the user device. The user device displays the received data in real time and converts it into a format that workers can use immediately.

[0178] Thus, the invention demonstrates how specific procedures, hardware, and software can be used to provide effective support methods and procedures for individuals and factory workers with disabilities.

[0179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0180] Step 1:

[0181] A user uses a terminal to input information on how to improve the efficiency of factory work and how to solve problems.

[0182] Input: Work task (e.g., "pick-and-place operation") and associated problem (e.g., "frequent machine stoppages").

[0183] Users use tablets or smart glasses to input information about the specific task or problem they are facing into the interface.

[0184] Step 2:

[0185] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using HTTPS.

[0186] Input: Work tasks and associated issues entered by the user.

[0187] Data is transmitted securely, ensuring consistency and integrity of information.

[0188] Step 3:

[0189] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and inputs it into a generative AI model.

[0190] Input: Structured data (work tasks and associated problems).

[0191] Data preprocessing transforms data into a format that is easy for generative AI models to analyze.

[0192] Step 4:

[0193] The server's generative AI model generates optimal work procedures and problem-solving methods based on the pre-processed data.

[0194] Input: Preprocessed data.

[0195] A generative AI model (e.g., OpenAI GPT series) performs analysis and generates efficient work procedures (e.g., "recalibrate sensors" or "introduce high-speed vision systems") and related cases.

[0196] Step 5:

[0197] The server restructures the generated work procedures and related cases and sends them to the terminal in JSON format.

[0198] Input: Generated work procedures and related cases.

[0199] The information is organized in a structured data format and converted into a form that is easy for end users to use.

[0200] Step 6:

[0201] The terminal interprets the received data and displays it in a visually easy-to-understand format for the user.

[0202] Input: Structured data sent by the server.

[0203] Users can check optimal work procedures and related examples in real time through tablets or smart glasses and immediately apply them to their work.

[0204] As an example, use the following prompt:

[0205] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[0206] This series of steps enables efficient work support through collaboration between the server and the terminal. This system is capable of seamlessly inputting information, generating optimal solutions, and then implementing them.

[0207] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0208] This invention is a system that provides individualized support to children with disabilities, and aims to improve the quality of support by using a generative AI model to propose optimal support methods and combining it with an emotion engine that recognizes the user's emotions. This system is mainly composed of a user terminal, a server, a generative AI model, and an emotion engine.

[0209] System Configuration

[0210] The system of the present invention comprises the following elements:

[0211] 1. User terminal: A device that provides an interface for teachers to input information and display the results.

[0212] 2. Server: Receives information from the user device and drives the generative AI model and emotion engine to generate and provide the optimal support method.

[0213] 3. Generative AI model: A machine learning algorithm that generates optimal assistance methods based on user input information and emotional data.

[0214] 4. Emotion Engine: An analytical engine that recognizes user emotions and provides feedback to the generative AI model.

[0215] Program processing

[0216] The system of the present invention performs processing in the following procedure.

[0217] User Input

[0218] The user (teacher) uses the terminal to input information about the child's disability or difficulties, such as "autism spectrum disorder" or "difficulty with social interaction."

[0219] Sending data

[0220] The terminal converts the entered information into a structured data format and sends it to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data.

[0221] Server reception and analysis

[0222] The server analyzes and preprocesses the data received from the device, including normalizing and tokenizing the text. It also analyzes the user's voice and facial expressions to obtain emotional data.

[0223] Running the Emotion Engine

[0224] The emotion engine detects emotions from the user's voice and facial expressions and feeds that data back to the generative AI model, so that the user's emotional state is reflected in the process of generating assistance methods.

[0225] Running generative AI

[0226] The server's generative AI model generates optimal support methods and related practical examples based on the input data and emotional data. For example, it suggests specific support methods such as "communication practice using picture cards."

[0227] Processing and sending results

[0228] The server organizes the generated support methods and practical examples into a data package, converts it back into a structured data format, and sends it to the terminal. This process also uses a secure protocol.

[0229] Displaying the results

[0230] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0231] Specific examples

[0232] For example, if a user inputs information such as "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting, this emotional data is fed back to the generation AI. Taking the user's emotional state into consideration, the generation AI suggests "communication practice using picture cards" and even adds advice such as "it is recommended to practice in an environment where the user can relax." This information is presented to the user via their device, allowing for prompt and appropriate support.

[0233] In this way, by combining emotion engines, it becomes possible to provide a more appropriate support method that takes into account the user's emotional state, thereby significantly improving the quality of support.

[0234] The processing flow will be explained below.

[0235] Step 1:

[0236] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[0237] Step 2:

[0238] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent to the server.

[0239] Step 3:

[0240] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[0241] Step 4:

[0242] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[0243] Step 5:

[0244] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[0245] Step 6:

[0246] The user activates the emotion engine, which analyzes the user's voice and facial expressions in real time and generates emotion data.

[0247] Step 7:

[0248] The server receives emotion data from the emotion engine and integrates it with the pre-processed data, which is then fed into the generative AI model.

[0249] Step 8:

[0250] The server inputs the integrated data into a generative AI model to generate optimal support methods and practical examples, such as "communication practice using picture cards."

[0251] Step 9:

[0252] The server organizes the generated support methods and practice examples into a data package, which includes details of the support methods and practice examples.

[0253] Step 10:

[0254] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[0255] Step 11:

[0256] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0257] Step 12:

[0258] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[0259] Example 2

[0260] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0261] When providing individualized support to children with disabilities, the effectiveness of the support can be reduced due to standardization of support methods or differences in implementation. Furthermore, the emotions and mental states of teachers and supporters affect the effectiveness of support methods, but conventional methods have had difficulty taking these into account. The present invention aims to solve these problems and improve the quality of support.

[0262] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0263] In this invention, the server includes means for generating an optimal support method using a generative AI model, means for acquiring emotional data using an emotion engine that analyzes the user's emotional data and feeding the data back to the generative AI model, and means for transmitting the support method generated using the generative AI model and practical examples to a terminal, thereby making it possible to provide a more appropriate support method according to each individual situation and emotional state.

[0264] "Disability" refers to a condition in which a person has limitations in physical or mental function, and has difficulty in carrying out daily life or social activities.

[0265] "Input means" refers to a device or interface for receiving information in digital form, and typically includes a keyboard, mouse, tablet, or touchscreen.

[0266] "Server" refers to a high-performance computer used for large amounts of data processing and calculations, and provides services to other computers and devices over a network.

[0267] "Generative AI model" means a system that is programmed based on machine learning algorithms and uses new data to automatically generate optimal assistance methods.

[0268] "Emotion engine" refers to software or hardware that analyzes emotions from data such as a user's voice, facial expressions, and text, and identifies the user's emotional state.

[0269] "Feedback means" refers to a mechanism or method for implementing a process that returns or provides specific information to another system or device.

[0270] A "structured data format" is a format in which data is organized in an orderly manner according to certain rules, and generally includes formats such as JSON, XML, and CSV.

[0271] "Secure protocol" refers to a communication protocol for ensuring security in data communication, and generally includes HTTPS, SSL / TLS, etc.

[0272] "Terminal" refers to an electronic device that is directly operated by a user, including, for example, a PC, tablet, or smartphone.

[0273] "Supportive measures" refers to the specific methods and procedures that help individuals with disabilities to carry out daily, academic, and social activities.

[0274] "Practical cases" refer to data showing records and results of specific support activities carried out in the past, which other users can use as reference.

[0275] "Preprocessing methods" refers to the processes and techniques used to transform raw data into a form suitable for analysis, including data cleaning, normalization, and tokenization.

[0276] This invention is a system that provides individualized support for children with disabilities. This system mainly consists of a user terminal, a server, a generative AI model, and an emotion engine.

[0277] Role of user terminal

[0278] The user (teacher) operates a device to input information about the child's disability or difficulties. The device is typically a PC or tablet. Input is done in form format, and specific information includes "autism spectrum disorder" and "difficulties with social interaction." After input, the device converts this information into a structured data format (e.g., JSON format) and securely sends it to the server via the HTTPS protocol.

[0279] Server Reception and Preprocessing

[0280] The server receives information sent from the user's device. After receiving the data, the server performs preprocessing such as text normalization and tokenization, and then analyzes the data. Preprocessing steps include removing unnecessary spaces and special characters, and segmenting words. The server also simultaneously collects voice and facial expression data provided by the user, and analyzes the emotional data using an emotion engine.

[0281] Running emotion engines and generative AI models

[0282] The server uses an emotion engine to identify the user's emotional state from their voice and facial expression data. This makes it possible to accurately detect, for example, whether the user is feeling stressed. This emotion data is fed back to the generative AI model. Based on the received input data and emotion data, the generative AI model generates optimal support methods and practical examples. As a specific example, it suggests a support method such as "communication practice using picture cards," and also adds the advice that "it is recommended to practice in a relaxed environment."

[0283] Sending and displaying results

[0284] The server organizes the generated support methods and practical examples and converts them back into structured data format. This data package is then securely sent to the user's device via the HTTPS protocol. The device retrieves the received data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen.

[0285] Examples and prompts

[0286] As a concrete example, consider a case where a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting. This emotion data is fed back to the generative AI model, which then suggests "communication practice using picture cards." It also adds the advice that "it is recommended to practice in a relaxing environment." This information is displayed on the device, allowing the user to provide prompt and appropriate support.

[0287] Prompt Sentence Examples

[0288] The input prompt for the generative AI model is:

[0289] "Disability: Autism Spectrum Disorder"

[0290] "Difficulties: Difficulties in social interactions"

[0291] "Emotion: Stress, Intensity: High"

[0292] The generative AI model then proposes optimal assistance methods based on these prompts, which can be directly input into the system and reflect the user's specific concerns and emotional state.

[0293] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0294] Program processing flow

[0295] Step 1:

[0296] User Input

[0297] The user (teacher) operates the device to input information about the child's disability or difficulties. The input method is a form format, and specific information such as "autism spectrum disorder" or "difficulty in social interaction" is entered into the device. The data obtained from this input becomes the initial input data for the system.

[0298] Step 2:

[0299] Sending data

[0300] The device converts the input information into a structured data format such as JSON, for example, "{\"Disability\": \"Autism Spectrum Disorder\", \"Problem\": \"Difficulty with Social Interaction\"}". The converted data is then sent to the server using the HTTPS protocol. Data integrity and confidentiality are ensured at this stage.

[0301] Step 3:

[0302] Server Reception and Preprocessing

[0303] The server receives the structured data sent from the device. It then performs preprocessing, normalization and tokenization, to remove unnecessary whitespace and special characters from the received data and to separate words. For example, the text "autism spectrum disorder" is divided into tokens such as "autism," "spectrum," and "disorder."

[0304] Step 4:

[0305] Acquiring user emotion data

[0306] The server receives the voice and facial expression data provided by the user at the same time. The server inputs this data into an emotion engine to analyze the user's emotional state. This analysis includes the tone and speaking rate of the voice data, and facial expression analysis of the facial expression data. For example, if the voice analysis detects high levels of "stress," this information is captured as emotion data.

[0307] Step 5:

[0308] Emotional Data Feedback

[0309] The server feeds back the acquired emotional data (e.g., "Emotion: Stress, Intensity: High") to the generative AI model. This feedback process allows the generative AI model to reflect the user's emotional state in the generation process of the assistance method.

[0310] Step 6:

[0311] Running generative AI models

[0312] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed input data and the fed-back emotional data. Using machine learning algorithms, the generative AI model suggests, for example, "communication practice using picture cards." It also provides advice such as "It is recommended to practice in an environment where the user can relax."

[0313] Step 7:

[0314] Processing and sending results

[0315] The server organizes the generated support methods and practice examples and converts them back into structured data in JSON format. The converted data package (e.g., "{\"Support method\": \"Communication practice using picture cards\", \"Recommended situation\": \"Perform in a relaxed environment\"}") is sent to the device using the HTTPS protocol.

[0316] Step 8:

[0317] Displaying the results

[0318] The device retrieves the data received from the server and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen. Based on this information, the user can quickly implement specific support methods.

[0319] (Application example 2)

[0320] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] Autonomous vehicles lack the ability to properly understand the emotional state of pedestrians and provide appropriate assistance based on that understanding. As a result, pedestrian safety and a sense of security cannot be fully ensured. Furthermore, conventional autonomous driving systems have difficulty adapting flexibly to the surrounding situation, and communication with pedestrians in particular is lacking.

[0322] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0323] In this invention, the server includes means for inputting information about the difficulties faced by an individual with a disability, means for transmitting the input information to the server, means for the server to receive the input information and search for an optimal assistance method using a generative AI model and an emotion engine, means for the server to transmit the assistance method and practical example generated using the generative AI model and emotion engine to a terminal, means for the terminal to display the transmitted assistance method and practical example, means for the system to collect video and audio data from a camera and microphone mounted on a mobile vehicle and analyze the emotional state from the data, and means for providing appropriate assistance methods to nearby users based on the emotional data analyzed by the generative AI model. This enables flexible and appropriate assistance to ensure safety and security by appropriately understanding the emotional state of pedestrians.

[0324] "Information on the challenges faced by individuals with disabilities" is data that documents the specific problems and difficulties faced by individuals with disabilities.

[0325] The "server" is a high-performance computer that processes data received from user terminals via the network and generates and distributes the necessary support methods.

[0326] A "generative AI model" is a machine learning algorithm that suggests optimal support methods based on information input by the user.

[0327] The "emotion engine" is a program that analyzes the user's emotions from video and audio data and provides appropriate feedback to the generative AI model.

[0328] A "terminal" is an electronic device that allows a user to input information and receive and display the generated support methods and practical examples.

[0329] "Mobile object" refers to a moving mechanical device such as an autonomous vehicle, equipped with sensors to sense its surroundings.

[0330] A "camera" is a photographic device used to collect video data and record the facial expressions and movements of pedestrians.

[0331] A "microphone" is a device for collecting audio data and is used to record surrounding sounds and speech.

[0332] "Video data" refers to visual information collected by a camera in the form of images or videos that are subject to analysis.

[0333] "Audio Data" means audio information collected by a microphone in the form of sound or speech that is subject to analysis.

[0334] The "means for analyzing emotional state" refers to a program or algorithm for determining the user's emotions based on video and audio data.

[0335] The "means for providing an appropriate support method" is a program or device for providing the user with the most appropriate response based on the analyzed emotional state.

[0336] A "structured data format" is a format for arranging data into a unified and easy-to-use format.

[0337] The specific system configuration and operation procedure for an embodiment of the present invention are described below. The system is broadly composed of a user terminal, a server, a generative AI model, an emotion engine, and a mobile object (an autonomously driven vehicle).

[0338] User terminal configuration and functions

[0339] A user terminal is an electronic device for inputting information about the difficulties faced by individuals with disabilities. The terminal provides an information input interface that allows users to input the type of disability and specific difficulties. Specifically, the terminal has an interface for typing information such as autism spectrum disorder and difficulties with social interaction.

[0340] Server configuration and functions

[0341] The server receives the information sent from the user device, analyzes the data, and executes the generative AI model. The server uses the following software in particular:

[0342] TENSORFLOW®: Running Generative AI Models

[0343] OpenCV: Video data preprocessing and analysis

[0344] EmotionAPI: Emotion Engine

[0345] The server analyzes the data sent from the user's device and generates optimal support methods and related practical examples using a generative AI model. Furthermore, it appropriately adjusts the generated support methods based on the emotional data analyzed by the emotion engine.

[0346] Linking generative AI models and emotion engines

[0347] The generative AI model is a machine learning algorithm that generates assistance methods based on information and emotional data input by the user. During this process, the emotion engine analyzes the user's emotional state and provides feedback to the generative AI model to provide more appropriate assistance methods.

[0348] Mobile device configuration and functions

[0349] This system uses cameras and microphones mounted on moving objects (autonomous vehicles) to collect video and audio data in real time. The emotion engine analyzes this data and determines the emotional state of pedestrians. The generative AI model provides appropriate assistance methods to nearby users based on the analyzed emotional data.

[0350] As a concrete example, consider a situation where a pedestrian stands at a crosswalk while an autonomous vehicle is traveling down the road. At this time, the system captures the pedestrian's facial expressions with a camera, and the emotion engine analyzes whether the pedestrian is feeling "tension" or "stress." Based on this emotional data, the generative AI model suggests "stopping the vehicle" and notifies the driver via a display and audio alert saying, "Please be careful, pedestrians have the right of way." This allows the pedestrian to cross the road safely.

[0351] Prompt Sentence Examples

[0352] Based on the input data from the camera and microphone installed in the autonomous vehicle, determine whether the pedestrian is feeling stressed and generate the optimal response method based on the result. For example, if the pedestrian is feeling stressed, consider a response such as "the vehicle stopping and encouraging the pedestrian to cross first."

[0353] As a result, the present invention enables autonomous vehicles to flexibly adapt to their surroundings, ensuring safety and a sense of security for pedestrians.

[0354] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0355] Step 1: Collect data

[0356] Cameras and microphones installed in autonomous vehicles collect video and audio data of the surrounding area in real time. The cameras capture pedestrians' faces and gestures, and the microphones record environmental sounds and pedestrians' voices. The collected video and audio data become input to the system.

[0357] Step 2: Preprocessing the data

[0358] The server uses OpenCV to identify faces and facial expressions from the collected video data. Specifically, the video is converted to grayscale and a face detection algorithm is applied. Meanwhile, the audio data is denoised and normalised to extract acoustic features. This pre-processing results in clean data for analysis.

[0359] Step 3: Sentiment Analysis

[0360] The server uses the Emotion API to analyze the pedestrian's emotional state from preprocessed video and audio data. The preprocessed video and audio are used as input, and the pedestrian's emotional state (e.g., tension, stress, relief) is returned as output. Specifically, emotions are determined from subtle changes in facial expressions and voice tone.

[0361] Step 4: Generating assistance methods using generative AI models

[0362] The server inputs the analyzed emotion data into a generative AI model using TensorFlow. This model generates the optimal assistance method by taking into account the pedestrian's emotion. The input is the pedestrian's emotion data, and the output is the appropriate countermeasure (e.g., stopping the vehicle, audio notification).

[0363] Step 5: Communicate how to help

[0364] The server sends the generated assistance method to the display and voice system of the autonomous vehicle. The display will show a message saying "Please be careful, pedestrians have the right of way," and the voice system will make a similar announcement. This allows the pedestrian to receive immediate and appropriate assistance.

[0365] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0366] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0367] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0368] [Second embodiment]

[0369] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0370] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0371] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0372] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0373] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0374] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0375] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0376] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0377] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0378] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0379] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0380] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0381] This invention is a system that supports individual support for children with disabilities. In particular, the system allows users (teachers) to input information about disabilities and difficulties, and uses a generative AI model to search for optimal support methods and practical examples, displaying the results on a terminal to provide fast and effective support.

[0382] System Configuration

[0383] This system mainly consists of the following components:

[0384] 1. User terminal: A device used by the teacher that provides an interface for entering information and displaying results.

[0385] 2. Server: Receives information from the user device, executes the generative AI model, and generates appropriate support methods and practical examples.

[0386] 3. Generative AI model: A machine learning algorithm that searches for and generates optimal support methods based on information about disabilities and difficulties.

[0387] Program processing

[0388] The program process for this system is as follows:

[0389] User Input

[0390] The user (teacher) uses the device interface to input information about the child's disability or difficulties, such as information about autism spectrum disorder or difficulties with social interaction.

[0391] Sending data

[0392] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using a secure protocol (HTTPS). At this stage, the consistency and integrity of the data is ensured.

[0393] Server reception and analysis

[0394] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and feeds it into a generative AI model. This preprocessing transforms the data into a format that is easier to analyze.

[0395] Running generative AI

[0396] The server's generative AI model searches for the optimal support method based on the input data and generates related practical examples, such as "communication support using picture cards."

[0397] Processing and sending results

[0398] The server then reorganizes the generated support methods and practical examples in a structured data format and sends it to the terminal. At this stage, the data is organized in a form that can be used immediately by the user.

[0399] Displaying the results

[0400] The device interprets the received data and displays it in a format that is easy for the user to understand. For example, it may visually present something like "Communication support methods for autism spectrum disorders" on the screen.

[0401] Specific examples

[0402] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," this information is sent to the server. The server uses generative AI to generate a result that "communication practice using picture cards" is effective, and sends this result along with appropriate examples to the device. As a result, the user can quickly obtain specific support methods and apply them in actual support.

[0403] As described above, the present invention specifically shows an embodiment for quickly and effectively providing an appropriate support method for children with disabilities.

[0404] The processing flow will be explained below.

[0405] Step 1:

[0406] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[0407] Step 2:

[0408] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent.

[0409] Step 3:

[0410] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[0411] Step 4:

[0412] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[0413] Step 5:

[0414] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[0415] Step 6:

[0416] The server inputs the preprocessed data into a generative AI model, which generates optimal support methods and practical examples based on the input data. For example, it outputs a specific support method such as "communication practice using picture cards."

[0417] Step 7:

[0418] The server organizes the output from the generative AI model into a data package, which includes details of assistance methods and practical examples.

[0419] Step 8:

[0420] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[0421] Step 9:

[0422] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0423] Step 10:

[0424] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[0425] Example 1

[0426] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0427] It is difficult to find a method for providing individualized support to children with disabilities quickly and effectively using conventional methods. In particular, they lack the ability to properly analyze input information and generate specific and practical support methods. There is also a need to display the generated information in a format that is easy for users to understand. The present invention aims to provide a system that solves these problems.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0429] In this invention, the server includes means for receiving input information and performing preprocessing such as text normalization and tokenization, means for generating optimal support methods and practical examples from the preprocessed information using a generative AI model, and means for transmitting the generated support methods and practical examples to the terminal in structured data format, thereby making it possible to quickly and effectively analyze the input information and provide specific and practical support methods to the user.

[0430] "Disability and difficulty information" is data that details the specific circumstances, problems, and difficulties of individuals with disabilities.

[0431] "Structured data" is a data format that is organized based on a specific format or standard, and is usually expressed in formats such as JSON or XML.

[0432] A "secure communication protocol" is a communication technology that prevents data exchange from being eavesdropped on or tampered with by third parties, and includes HTTPS.

[0433] "Preprocessing" refers to processes such as text normalization and tokenization to make data easier for generative AI models to parse.

[0434] "Normalization" refers to the process of organizing text data by removing unnecessary symbols and spaces.

[0435] "Tokenization" refers to the process of dividing text data into units such as words and phrases.

[0436] A "generative AI model" is a model that uses machine learning algorithms to generate optimal support methods and practical examples based on input data.

[0437] "Assistive techniques" refers to specific aids or approaches provided to individuals with disabilities.

[0438] "Practical cases" are data that describe specific successful cases and methods of support that have been provided in the past.

[0439] A "structured data format" is a form of data that is systematically organized in a specific format, commonly including JSON and XML.

[0440] A "terminal" is a device, such as a computer, tablet, or smartphone, that a user uses to enter information and view results.

[0441] The "format that is easy for users to understand" is a format in which the generated support methods and practical examples are displayed in a visually easy-to-understand manner so that the user can easily understand them.

[0442] This invention is a system that provides individualized support to children with disabilities quickly and effectively. This system mainly consists of a user terminal, a server, and a generative AI model.

[0443] User terminal

[0444] User terminals are devices used by teachers and supporters, and provide an interface for inputting information about children's disabilities and difficulties. Specifically, PCs, tablets, smartphones, etc. are used. The terminals are equipped with a dedicated form where users can enter various information, and the information entered through this form is converted into structured data (e.g., JSON format). This structured data is sent to the server using a secure communication protocol such as HTTPS.

[0445] server

[0446] The server receives the information sent from the user device and performs preprocessing on the data. Specific preprocessing includes normalizing and tokenizing the text. For example, it removes unnecessary symbols and spaces from the text and checks the integrity of the data. This preprocessing prepares the data in a format that is easy for the generative AI model to analyze.

[0447] The server uses a generative AI model to generate optimal support methods and practical examples from the preprocessed data. The generated support methods and practical examples are then organized again in a structured data format and sent to the user's device. At this time, the generated information is prepared to be displayed in a visually easy-to-understand format.

[0448] Generative AI Models

[0449] The generative AI model uses machine learning algorithms to generate optimal support methods and practical examples based on input data. This server-based model analyzes detailed information about disabilities and problems, and suggests optimal support methods based on past support data.

[0450] Specific examples

[0451] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction" on a device, this information is converted into JSON format and sent to the server using the secure HTTPS protocol. The server receives this information and normalizes and tokenizes the text. The generative AI model then analyzes this data and generates a "communication support method using picture cards" and specific practical examples. This generated information is then sent back to the device in structured data format and displayed in a format that is easy for the user to understand.

[0452] Prompt Sentence Examples

[0453] Examples of prompts that users can input to a generative AI model include:

[0454] "When a user inputs the information 'autism spectrum disorder' and 'difficulty in social interaction,' please generate the optimal assistance method."

[0455] Thus, the present invention specifically illustrates an embodiment for providing individualized support to children with disabilities quickly and effectively.

[0456] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0457] Step 1:

[0458] The user (teacher) uses the terminal interface to input information about the child's disability or difficulties. Specifically, the user enters information such as "autism spectrum disorder" or "difficulty in social interaction" into an input form. This input is provided by the user in the form of text data.

[0459] Input: Text data about a child's disability or difficulties (e.g., "autism spectrum disorder" or "difficulty with social interaction")

[0460] Output: Structured data (JSON format)

[0461] Step 2:

[0462] The terminal converts the information entered by the user into structured data (JSON format), which organizes the data in a consistent format, and then transmits the converted data to the server using a secure communication protocol (HTTPS).

[0463] Input: Text data (information entered by the user)

[0464] Output: Structured data (JSON format), submitted data

[0465] Step 3:

[0466] The server receives the structured data sent from the terminal. The received data is first preprocessed, such as text normalization and tokenization, to remove unnecessary symbols and whitespace to ensure data consistency and integrity.

[0467] Input: Structured data (JSON format)

[0468] Output: Preprocessed data (text normalized, tokenized data)

[0469] Step 4:

[0470] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed data. The generative AI model uses machine learning algorithms to analyze the data and generate optimal solutions by referring to past case data. In this process, new support methods and practical examples are created based on the data entered by the user.

[0471] Input: Preprocessed data (text normalized, tokenized data)

[0472] Output: Support methods and practical examples

[0473] Step 5:

[0474] The server then organizes the generated support methods and practical examples in a structured data format (JSON format) and sends it to the user's device. At this stage, the data is formatted so that it can be displayed visually in an easy-to-understand manner.

[0475] Input: Support methods and practical examples

[0476] Output: Structured data (JSON format), submitted structured data

[0477] Step 6:

[0478] The device interprets the structured data received from the server and displays it in a format that is easy for the user to understand. Specifically, it visually presents specific support methods and practical examples along with titles such as "Communication Support Methods for Autism Spectrum Disorder."

[0479] Input: Structured data (JSON format), data received from the server

[0480] Output: A visually easy-to-understand display for the user

[0481] At each step, data processing and calculations are performed based on the input and output to ensure the smooth functioning of the entire system, and the system is designed to enable users to obtain assistance methods quickly and effectively.

[0482] (Application example 1)

[0483] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0484] In addition to providing appropriate assistance methods for individuals with disabilities, there is a need for a system that quickly provides efficient work procedures and problem-solving methods to factory workers. However, because these assistance methods and work procedures vary depending on individual circumstances, a one-size-fits-all solution is insufficient. Furthermore, delays in providing real-time assistance methods and work procedures can lead to reduced productivity and work efficiency.

[0485] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0486] In this invention, the server includes: means for inputting information about a problem faced by an individual with a disability; means for transmitting the input information to the server; means for the server receiving the input information and searching for an optimal assistance method using a generative AI model; means for the server transmitting the assistance method generated using the generative AI model and practical examples to a terminal; means for the terminal displaying the transmitted assistance method and practical examples; means for the terminal inputting a factory work efficiency improvement method or problem-solving method and generating an optimal work procedure using a generative AI model; means for the server transmitting the generated work procedure and related examples to the terminal; and means for the terminal displaying the transmitted work procedure and related examples. This makes it possible to provide not only appropriate and personalized assistance methods for individuals with disabilities, but also efficient work procedures and problem-solving methods in real time to factory workers.

[0487] "Individual with a disability" means a person who has a physical or mental impairment.

[0488] "Information about problems" refers to data about specific challenges and problems that individuals face in their daily lives and at work.

[0489] "Server" refers to a computer system that receives, processes, and transmits information over a network.

[0490] A "generative AI model" refers to an artificial intelligence system that uses machine learning algorithms to automatically generate optimal solutions and procedures.

[0491] "Optimal support methods" refer to the most effective and appropriate responses and support methods for an individual's problems.

[0492] "Practical examples" refer to examples of support methods or solutions that have been shown to be effective based on specific situations or cases.

[0493] A "terminal" is a device that allows users to input and output data, and in this case refers to tablets and smart glasses used by factory workers.

[0494] "Improving factory work efficiency" refers to improving productivity by optimizing work procedures and the work environment in a factory.

[0495] A "problem-solving method" refers to a means or procedure for solving a specific problem or obstacle.

[0496] "Work procedures" refer to the specific steps and methods for performing work or operations within a factory.

[0497] "Relevant examples" refer to specific examples or case studies of successful implementations of similar situations or challenges.

[0498] This invention is a system that quickly provides appropriate support methods and efficiency procedures for individuals and factory workers with disabilities. It mainly consists of the following specific components:

[0499] 1. User Device:

[0500] This is a device for inputting information about the difficulties faced by individuals with disabilities and the efficiency of factory work. This includes devices such as tablets and smart glasses. For example, workers can input information about "pick-and-place operations."

[0501] 2. Server:

[0502] This is a computer system that receives and preprocesses information sent from the user's device. This server contains a generative AI model and automatically generates optimal support methods and work procedures based on the input information. Cloud-based servers such as AWS EC2 and Google Cloud Platform are often used.

[0503] 3. Generative AI Model:

[0504] These AI systems use machine learning algorithms to generate optimal solutions based on input information, such as OpenAI's GPT series. These AI models can provide personalized assistance for individuals with disabilities or optimal work procedures for factory workers.

[0505] Example

[0506] Improving factory work efficiency

[0507] Users input information about issues with "pick-and-place operations" and "frequent machine stoppages" via a tablet. Based on this, the server uses a generative AI model to generate optimal work procedures, such as "recalibrating sensors" and "introducing a high-speed vision system." The generated procedures and related cases are sent to the terminal, allowing workers to take immediate action.

[0508] For example, enter the following prompt:

[0509] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[0510] Hardware and software used

[0511] Hardware:

[0512] Smart glasses (e.g. Microsoft HoloLens)

[0513] Tablets (e.g. iPad Pro)

[0514] Factory robots (e.g. ABB IRB series, KUKA KR series)

[0515] software:

[0516] Cloud servers (e.g. AWS EC2, Google Cloud Platform)

[0517] Machine learning API (e.g. OpenAI GPT series)

[0518] Data processing and calculation

[0519] The server first preprocesses the information received from the user device. This preprocessing includes text normalization and tokenization. The generative AI model then generates optimal assistance methods and work procedures based on the preprocessed data. This generated data is then restructured and sent to the user device. The user device displays the received data in real time and converts it into a format that workers can use immediately.

[0520] Thus, the invention demonstrates how specific procedures, hardware, and software can be used to provide effective support methods and procedures for individuals and factory workers with disabilities.

[0521] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0522] Step 1:

[0523] A user uses a terminal to input information on how to improve the efficiency of factory work and how to solve problems.

[0524] Input: Work task (e.g., "pick-and-place operation") and associated problem (e.g., "frequent machine stoppages").

[0525] Users use tablets or smart glasses to input information about the specific task or problem they are facing into the interface.

[0526] Step 2:

[0527] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using HTTPS.

[0528] Input: Work tasks and associated issues entered by the user.

[0529] Data is transmitted securely, ensuring consistency and integrity of information.

[0530] Step 3:

[0531] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and inputs it into a generative AI model.

[0532] Input: Structured data (work tasks and associated problems).

[0533] Data preprocessing transforms data into a format that is easy for generative AI models to analyze.

[0534] Step 4:

[0535] The server's generative AI model generates optimal work procedures and problem-solving methods based on the pre-processed data.

[0536] Input: Preprocessed data.

[0537] A generative AI model (e.g., OpenAI GPT series) performs analysis and generates efficient work procedures (e.g., "recalibrate sensors" or "introduce high-speed vision systems") and related cases.

[0538] Step 5:

[0539] The server restructures the generated work procedures and related cases and sends them to the terminal in JSON format.

[0540] Input: Generated work procedures and related cases.

[0541] The information is organized in a structured data format and converted into a form that is easy for end users to use.

[0542] Step 6:

[0543] The terminal interprets the received data and displays it in a visually easy-to-understand format for the user.

[0544] Input: Structured data sent by the server.

[0545] Users can check optimal work procedures and related examples in real time through tablets or smart glasses and immediately apply them to their work.

[0546] As an example, use the following prompt:

[0547] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[0548] This series of steps enables efficient work support through collaboration between the server and the terminal. This system is capable of seamlessly inputting information, generating optimal solutions, and then implementing them.

[0549] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0550] This invention is a system that provides individualized support to children with disabilities, and aims to improve the quality of support by using a generative AI model to propose optimal support methods and combining it with an emotion engine that recognizes the user's emotions. This system is mainly composed of a user terminal, a server, a generative AI model, and an emotion engine.

[0551] System Configuration

[0552] The system of the present invention comprises the following elements:

[0553] 1. User terminal: A device that provides an interface for teachers to input information and display the results.

[0554] 2. Server: Receives information from the user device and drives the generative AI model and emotion engine to generate and provide the optimal support method.

[0555] 3. Generative AI model: A machine learning algorithm that generates optimal assistance methods based on user input information and emotional data.

[0556] 4. Emotion Engine: An analytical engine that recognizes user emotions and provides feedback to the generative AI model.

[0557] Program processing

[0558] The system of the present invention performs processing in the following procedure.

[0559] User Input

[0560] The user (teacher) uses the terminal to input information about the child's disability or difficulties, such as "autism spectrum disorder" or "difficulty with social interaction."

[0561] Sending data

[0562] The terminal converts the entered information into a structured data format and sends it to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data.

[0563] Server reception and analysis

[0564] The server analyzes and preprocesses the data received from the device, including normalizing and tokenizing the text. It also analyzes the user's voice and facial expressions to obtain emotional data.

[0565] Running the Emotion Engine

[0566] The emotion engine detects emotions from the user's voice and facial expressions and feeds that data back to the generative AI model, so that the user's emotional state is reflected in the process of generating assistance methods.

[0567] Running generative AI

[0568] The server's generative AI model generates optimal support methods and related practical examples based on the input data and emotional data. For example, it suggests specific support methods such as "communication practice using picture cards."

[0569] Processing and sending results

[0570] The server organizes the generated support methods and practical examples into a data package, converts it back into a structured data format, and sends it to the terminal. This process also uses a secure protocol.

[0571] Displaying the results

[0572] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0573] Specific examples

[0574] For example, if a user inputs information such as "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting, this emotional data is fed back to the generation AI. Taking the user's emotional state into consideration, the generation AI suggests "communication practice using picture cards" and even adds advice such as "it is recommended to practice in an environment where the user can relax." This information is presented to the user via their device, allowing for prompt and appropriate support.

[0575] In this way, by combining emotion engines, it becomes possible to provide a more appropriate support method that takes into account the user's emotional state, thereby significantly improving the quality of support.

[0576] The processing flow will be explained below.

[0577] Step 1:

[0578] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[0579] Step 2:

[0580] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent to the server.

[0581] Step 3:

[0582] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[0583] Step 4:

[0584] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[0585] Step 5:

[0586] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[0587] Step 6:

[0588] The user activates the emotion engine, which analyzes the user's voice and facial expressions in real time and generates emotion data.

[0589] Step 7:

[0590] The server receives emotion data from the emotion engine and integrates it with the pre-processed data, which is then fed into the generative AI model.

[0591] Step 8:

[0592] The server inputs the integrated data into a generative AI model to generate optimal support methods and practical examples, such as "communication practice using picture cards."

[0593] Step 9:

[0594] The server organizes the generated support methods and practice examples into a data package, which includes details of the support methods and practice examples.

[0595] Step 10:

[0596] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[0597] Step 11:

[0598] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0599] Step 12:

[0600] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[0601] Example 2

[0602] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0603] When providing individualized support to children with disabilities, the effectiveness of the support can be reduced due to standardization of support methods or differences in implementation. Furthermore, the emotions and mental states of teachers and supporters affect the effectiveness of support methods, but conventional methods have had difficulty taking these into account. The present invention aims to solve these problems and improve the quality of support.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0605] In this invention, the server includes means for generating an optimal support method using a generative AI model, means for acquiring emotional data using an emotion engine that analyzes the user's emotional data and feeding the data back to the generative AI model, and means for transmitting the support method generated using the generative AI model and practical examples to a terminal, thereby making it possible to provide a more appropriate support method according to each individual situation and emotional state.

[0606] "Disability" refers to a condition in which a person has limitations in physical or mental function, and has difficulty in carrying out daily life or social activities.

[0607] "Input means" refers to a device or interface for receiving information in digital form, and typically includes a keyboard, mouse, tablet, or touchscreen.

[0608] "Server" refers to a high-performance computer used for large amounts of data processing and calculations, and provides services to other computers and devices over a network.

[0609] "Generative AI model" means a system that is programmed based on machine learning algorithms and uses new data to automatically generate optimal assistance methods.

[0610] "Emotion engine" refers to software or hardware that analyzes emotions from data such as a user's voice, facial expressions, and text, and identifies the user's emotional state.

[0611] "Feedback means" refers to a mechanism or method for implementing a process that returns or provides specific information to another system or device.

[0612] A "structured data format" is a format in which data is organized in an orderly manner according to certain rules, and generally includes formats such as JSON, XML, and CSV.

[0613] "Secure protocol" refers to a communication protocol for ensuring security in data communication, and generally includes HTTPS, SSL / TLS, etc.

[0614] "Terminal" refers to an electronic device that is directly operated by a user, including, for example, a PC, tablet, or smartphone.

[0615] "Supportive measures" refers to the specific methods and procedures that help individuals with disabilities to carry out daily, academic, and social activities.

[0616] "Practical cases" refer to data showing records and results of specific support activities carried out in the past, which other users can use as reference.

[0617] "Preprocessing methods" refers to the processes and techniques used to transform raw data into a form suitable for analysis, including data cleaning, normalization, and tokenization.

[0618] This invention is a system that provides individualized support for children with disabilities. This system mainly consists of a user terminal, a server, a generative AI model, and an emotion engine.

[0619] Role of user terminal

[0620] The user (teacher) operates a device to input information about the child's disability or difficulties. The device is typically a PC or tablet. Input is done in form format, and specific information includes "autism spectrum disorder" and "difficulties with social interaction." After input, the device converts this information into a structured data format (e.g., JSON format) and securely sends it to the server via the HTTPS protocol.

[0621] Server Reception and Preprocessing

[0622] The server receives information sent from the user's device. After receiving the data, the server performs preprocessing such as text normalization and tokenization, and then analyzes the data. Preprocessing steps include removing unnecessary spaces and special characters, and segmenting words. The server also simultaneously collects voice and facial expression data provided by the user, and analyzes the emotional data using an emotion engine.

[0623] Running emotion engines and generative AI models

[0624] The server uses an emotion engine to identify the user's emotional state from their voice and facial expression data. This makes it possible to accurately detect, for example, whether the user is feeling stressed. This emotion data is fed back to the generative AI model. Based on the received input data and emotion data, the generative AI model generates optimal support methods and practical examples. As a specific example, it suggests a support method such as "communication practice using picture cards," and also adds the advice that "it is recommended to practice in a relaxed environment."

[0625] Sending and displaying results

[0626] The server organizes the generated support methods and practical examples and converts them back into structured data format. This data package is then securely sent to the user's device via the HTTPS protocol. The device retrieves the received data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen.

[0627] Examples and prompts

[0628] As a concrete example, consider a case where a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting. This emotion data is fed back to the generative AI model, which then suggests "communication practice using picture cards." It also adds the advice that "it is recommended to practice in a relaxing environment." This information is displayed on the device, allowing the user to provide prompt and appropriate support.

[0629] Prompt Sentence Examples

[0630] The input prompt for the generative AI model is:

[0631] "Disability: Autism Spectrum Disorder"

[0632] "Difficulties: Difficulties in social interactions"

[0633] "Emotion: Stress, Intensity: High"

[0634] The generative AI model then proposes optimal assistance methods based on these prompts, which can be directly input into the system and reflect the user's specific concerns and emotional state.

[0635] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0636] Program processing flow

[0637] Step 1:

[0638] User Input

[0639] The user (teacher) operates the device to input information about the child's disability or difficulties. The input method is a form format, and specific information such as "autism spectrum disorder" or "difficulty in social interaction" is entered into the device. The data obtained from this input becomes the initial input data for the system.

[0640] Step 2:

[0641] Sending data

[0642] The device converts the input information into a structured data format such as JSON, for example, "{\"Disability\": \"Autism Spectrum Disorder\", \"Problem\": \"Difficulty with Social Interaction\"}". The converted data is then sent to the server using the HTTPS protocol. Data integrity and confidentiality are ensured at this stage.

[0643] Step 3:

[0644] Server Reception and Preprocessing

[0645] The server receives the structured data sent from the device. It then performs preprocessing, normalization and tokenization, to remove unnecessary whitespace and special characters from the received data and to separate words. For example, the text "autism spectrum disorder" is divided into tokens such as "autism," "spectrum," and "disorder."

[0646] Step 4:

[0647] Acquiring user emotion data

[0648] The server receives the voice and facial expression data provided by the user at the same time. The server inputs this data into an emotion engine to analyze the user's emotional state. This analysis includes the tone and speaking rate of the voice data, and facial expression analysis of the facial expression data. For example, if the voice analysis detects high levels of "stress," this information is captured as emotion data.

[0649] Step 5:

[0650] Emotional Data Feedback

[0651] The server feeds back the acquired emotional data (e.g., "Emotion: Stress, Intensity: High") to the generative AI model. This feedback process allows the generative AI model to reflect the user's emotional state in the generation process of the assistance method.

[0652] Step 6:

[0653] Running generative AI models

[0654] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed input data and the fed-back emotional data. Using machine learning algorithms, the generative AI model suggests, for example, "communication practice using picture cards." It also provides advice such as "It is recommended to practice in an environment where the user can relax."

[0655] Step 7:

[0656] Processing and sending results

[0657] The server organizes the generated support methods and practice examples and converts them back into structured data in JSON format. The converted data package (e.g., "{\"Support method\": \"Communication practice using picture cards\", \"Recommended situation\": \"Perform in a relaxed environment\"}") is sent to the device using the HTTPS protocol.

[0658] Step 8:

[0659] Displaying the results

[0660] The device retrieves the data received from the server and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen. Based on this information, the user can quickly implement specific support methods.

[0661] (Application example 2)

[0662] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0663] Autonomous vehicles lack the ability to properly understand the emotional state of pedestrians and provide appropriate assistance based on that understanding. As a result, pedestrian safety and a sense of security cannot be fully ensured. Furthermore, conventional autonomous driving systems have difficulty adapting flexibly to the surrounding situation, and communication with pedestrians in particular is lacking.

[0664] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0665] In this invention, the server includes means for inputting information about the difficulties faced by an individual with a disability, means for transmitting the input information to the server, means for the server to receive the input information and search for an optimal assistance method using a generative AI model and an emotion engine, means for the server to transmit the assistance method and practical example generated using the generative AI model and emotion engine to a terminal, means for the terminal to display the transmitted assistance method and practical example, means for the system to collect video and audio data from a camera and microphone mounted on a mobile vehicle and analyze the emotional state from the data, and means for providing appropriate assistance methods to nearby users based on the emotional data analyzed by the generative AI model. This enables flexible and appropriate assistance to ensure safety and security by appropriately understanding the emotional state of pedestrians.

[0666] "Information on the challenges faced by individuals with disabilities" is data that documents the specific problems and difficulties faced by individuals with disabilities.

[0667] The "server" is a high-performance computer that processes data received from user terminals via the network and generates and distributes the necessary support methods.

[0668] A "generative AI model" is a machine learning algorithm that suggests optimal support methods based on information input by the user.

[0669] The "emotion engine" is a program that analyzes the user's emotions from video and audio data and provides appropriate feedback to the generative AI model.

[0670] A "terminal" is an electronic device that allows a user to input information and receive and display the generated support methods and practical examples.

[0671] "Mobile object" refers to a moving mechanical device such as an autonomous vehicle, equipped with sensors to sense its surroundings.

[0672] A "camera" is a photographic device used to collect video data and record the facial expressions and movements of pedestrians.

[0673] A "microphone" is a device for collecting audio data and is used to record surrounding sounds and speech.

[0674] "Video data" refers to visual information collected by a camera in the form of images or videos that are subject to analysis.

[0675] "Audio Data" means audio information collected by a microphone in the form of sound or speech that is subject to analysis.

[0676] The "means for analyzing emotional state" refers to a program or algorithm for determining the user's emotions based on video and audio data.

[0677] The "means for providing an appropriate support method" is a program or device for providing the user with the most appropriate response based on the analyzed emotional state.

[0678] A "structured data format" is a format for arranging data into a unified and easy-to-use format.

[0679] The specific system configuration and operation procedure for an embodiment of the present invention are described below. The system is broadly composed of a user terminal, a server, a generative AI model, an emotion engine, and a mobile object (an autonomously driven vehicle).

[0680] User terminal configuration and functions

[0681] A user terminal is an electronic device for inputting information about the difficulties faced by individuals with disabilities. The terminal provides an information input interface that allows users to input the type of disability and specific difficulties. Specifically, the terminal has an interface for typing information such as autism spectrum disorder and difficulties with social interaction.

[0682] Server configuration and functions

[0683] The server receives the information sent from the user device, analyzes the data, and executes the generative AI model. The server uses the following software in particular:

[0684] TensorFlow: Running Generative AI Models

[0685] OpenCV: Video data preprocessing and analysis

[0686] EmotionAPI: Emotion Engine

[0687] The server analyzes the data sent from the user's device and generates optimal support methods and related practical examples using a generative AI model. Furthermore, it appropriately adjusts the generated support methods based on the emotional data analyzed by the emotion engine.

[0688] Linking generative AI models and emotion engines

[0689] The generative AI model is a machine learning algorithm that generates assistance methods based on information and emotional data input by the user. During this process, the emotion engine analyzes the user's emotional state and provides feedback to the generative AI model to provide more appropriate assistance methods.

[0690] Mobile device configuration and functions

[0691] This system uses cameras and microphones mounted on moving objects (autonomous vehicles) to collect video and audio data in real time. The emotion engine analyzes this data and determines the emotional state of pedestrians. The generative AI model provides appropriate assistance methods to nearby users based on the analyzed emotional data.

[0692] As a concrete example, consider a situation where a pedestrian stands at a crosswalk while an autonomous vehicle is traveling down the road. At this time, the system captures the pedestrian's facial expressions with a camera, and the emotion engine analyzes whether the pedestrian is feeling "tension" or "stress." Based on this emotional data, the generative AI model suggests "stopping the vehicle" and notifies the driver via a display and audio alert saying, "Please be careful, pedestrians have the right of way." This allows the pedestrian to cross the road safely.

[0693] Prompt Sentence Examples

[0694] Based on the input data from the camera and microphone installed in the autonomous vehicle, determine whether the pedestrian is feeling stressed and generate the optimal response method based on the result. For example, if the pedestrian is feeling stressed, consider a response such as "the vehicle stopping and encouraging the pedestrian to cross first."

[0695] As a result, the present invention enables autonomous vehicles to flexibly adapt to their surroundings, ensuring safety and a sense of security for pedestrians.

[0696] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0697] Step 1: Collect data

[0698] Cameras and microphones installed in autonomous vehicles collect video and audio data of the surrounding area in real time. The cameras capture pedestrians' faces and gestures, and the microphones record environmental sounds and pedestrians' voices. The collected video and audio data become input to the system.

[0699] Step 2: Preprocessing the data

[0700] The server uses OpenCV to identify faces and facial expressions from the collected video data. Specifically, the video is converted to grayscale and a face detection algorithm is applied. Meanwhile, the audio data is denoised and normalised to extract acoustic features. This pre-processing results in clean data for analysis.

[0701] Step 3: Sentiment Analysis

[0702] The server uses the Emotion API to analyze the pedestrian's emotional state from preprocessed video and audio data. The preprocessed video and audio are used as input, and the pedestrian's emotional state (e.g., tension, stress, relief) is returned as output. Specifically, emotions are determined from subtle changes in facial expressions and voice tone.

[0703] Step 4: Generating assistance methods using generative AI models

[0704] The server inputs the analyzed emotion data into a generative AI model using TensorFlow. This model generates the optimal assistance method by taking into account the pedestrian's emotion. The input is the pedestrian's emotion data, and the output is the appropriate countermeasure (e.g., stopping the vehicle, audio notification).

[0705] Step 5: Communicate how to help

[0706] The server sends the generated assistance method to the display and voice system of the autonomous vehicle. The display will show a message saying "Please be careful, pedestrians have the right of way," and the voice system will make a similar announcement. This allows the pedestrian to receive immediate and appropriate assistance.

[0707] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0708] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0709] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0710] [Third embodiment]

[0711] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0712] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0713] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0714] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0715] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0716] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0717] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0718] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0719] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0720] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0721] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0722] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0723] This invention is a system that supports individual support for children with disabilities. In particular, the system allows users (teachers) to input information about disabilities and difficulties, and uses a generative AI model to search for optimal support methods and practical examples, displaying the results on a terminal to provide fast and effective support.

[0724] System Configuration

[0725] This system mainly consists of the following components:

[0726] 1. User terminal: A device used by the teacher that provides an interface for entering information and displaying results.

[0727] 2. Server: Receives information from the user device, executes the generative AI model, and generates appropriate support methods and practical examples.

[0728] 3. Generative AI model: A machine learning algorithm that searches for and generates optimal support methods based on information about disabilities and difficulties.

[0729] Program processing

[0730] The program process for this system is as follows:

[0731] User Input

[0732] The user (teacher) uses the device interface to input information about the child's disability or difficulties, such as information about autism spectrum disorder or difficulties with social interaction.

[0733] Sending data

[0734] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using a secure protocol (HTTPS). At this stage, the consistency and integrity of the data is ensured.

[0735] Server reception and analysis

[0736] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and feeds it into a generative AI model. This preprocessing transforms the data into a format that is easier to analyze.

[0737] Running generative AI

[0738] The server's generative AI model searches for the optimal support method based on the input data and generates related practical examples, such as "communication support using picture cards."

[0739] Processing and sending results

[0740] The server then reorganizes the generated support methods and practical examples in a structured data format and sends it to the terminal. At this stage, the data is organized in a form that can be used immediately by the user.

[0741] Displaying the results

[0742] The device interprets the received data and displays it in a format that is easy for the user to understand. For example, it may visually present something like "Communication support methods for autism spectrum disorders" on the screen.

[0743] Specific examples

[0744] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," this information is sent to the server. The server uses generative AI to generate a result that "communication practice using picture cards" is effective, and sends this result along with appropriate examples to the device. As a result, the user can quickly obtain specific support methods and apply them in actual support.

[0745] As described above, the present invention specifically shows an embodiment for quickly and effectively providing an appropriate support method for children with disabilities.

[0746] The processing flow will be explained below.

[0747] Step 1:

[0748] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[0749] Step 2:

[0750] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent.

[0751] Step 3:

[0752] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[0753] Step 4:

[0754] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[0755] Step 5:

[0756] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[0757] Step 6:

[0758] The server inputs the preprocessed data into a generative AI model, which generates optimal support methods and practical examples based on the input data. For example, it outputs a specific support method such as "communication practice using picture cards."

[0759] Step 7:

[0760] The server organizes the output from the generative AI model into a data package, which includes details of assistance methods and practical examples.

[0761] Step 8:

[0762] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[0763] Step 9:

[0764] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0765] Step 10:

[0766] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[0767] Example 1

[0768] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0769] It is difficult to find a method for providing individualized support to children with disabilities quickly and effectively using conventional methods. In particular, they lack the ability to properly analyze input information and generate specific and practical support methods. There is also a need to display the generated information in a format that is easy for users to understand. The present invention aims to provide a system that solves these problems.

[0770] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0771] In this invention, the server includes means for receiving input information and performing preprocessing such as text normalization and tokenization, means for generating optimal support methods and practical examples from the preprocessed information using a generative AI model, and means for transmitting the generated support methods and practical examples to the terminal in structured data format, thereby making it possible to quickly and effectively analyze the input information and provide specific and practical support methods to the user.

[0772] "Disability and difficulty information" is data that details the specific circumstances, problems, and difficulties of individuals with disabilities.

[0773] "Structured data" is a data format that is organized based on a specific format or standard, and is usually expressed in formats such as JSON or XML.

[0774] A "secure communication protocol" is a communication technology that prevents data exchange from being eavesdropped on or tampered with by third parties, and includes HTTPS.

[0775] "Preprocessing" refers to processes such as text normalization and tokenization to make data easier for generative AI models to parse.

[0776] "Normalization" refers to the process of organizing text data by removing unnecessary symbols and spaces.

[0777] "Tokenization" refers to the process of dividing text data into units such as words and phrases.

[0778] A "generative AI model" is a model that uses machine learning algorithms to generate optimal support methods and practical examples based on input data.

[0779] "Assistive techniques" refers to specific aids or approaches provided to individuals with disabilities.

[0780] "Practical cases" are data that describe specific successful cases and methods of support that have been provided in the past.

[0781] A "structured data format" is a form of data that is systematically organized in a specific format, commonly including JSON and XML.

[0782] A "terminal" is a device, such as a computer, tablet, or smartphone, that a user uses to enter information and view results.

[0783] The "format that is easy for users to understand" is a format in which the generated support methods and practical examples are displayed in a visually easy-to-understand manner so that the user can easily understand them.

[0784] This invention is a system that provides individualized support to children with disabilities quickly and effectively. This system mainly consists of a user terminal, a server, and a generative AI model.

[0785] User terminal

[0786] User terminals are devices used by teachers and supporters, and provide an interface for inputting information about children's disabilities and difficulties. Specifically, PCs, tablets, smartphones, etc. are used. The terminals are equipped with a dedicated form where users can enter various information, and the information entered through this form is converted into structured data (e.g., JSON format). This structured data is sent to the server using a secure communication protocol such as HTTPS.

[0787] server

[0788] The server receives the information sent from the user device and performs preprocessing on the data. Specific preprocessing includes normalizing and tokenizing the text. For example, it removes unnecessary symbols and spaces from the text and checks the integrity of the data. This preprocessing prepares the data in a format that is easy for the generative AI model to analyze.

[0789] The server uses a generative AI model to generate optimal support methods and practical examples from the preprocessed data. The generated support methods and practical examples are then organized again in a structured data format and sent to the user's device. At this time, the generated information is prepared to be displayed in a visually easy-to-understand format.

[0790] Generative AI Models

[0791] The generative AI model uses machine learning algorithms to generate optimal support methods and practical examples based on input data. This server-based model analyzes detailed information about disabilities and problems, and suggests optimal support methods based on past support data.

[0792] Specific examples

[0793] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction" on a device, this information is converted into JSON format and sent to the server using the secure HTTPS protocol. The server receives this information and normalizes and tokenizes the text. The generative AI model then analyzes this data and generates a "communication support method using picture cards" and specific practical examples. This generated information is then sent back to the device in structured data format and displayed in a format that is easy for the user to understand.

[0794] Prompt Sentence Examples

[0795] Examples of prompts that users can input to a generative AI model include:

[0796] "When a user inputs the information 'autism spectrum disorder' and 'difficulty in social interaction,' please generate the optimal assistance method."

[0797] Thus, the present invention specifically illustrates an embodiment for providing individualized support to children with disabilities quickly and effectively.

[0798] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0799] Step 1:

[0800] The user (teacher) uses the terminal interface to input information about the child's disability or difficulties. Specifically, the user enters information such as "autism spectrum disorder" or "difficulty in social interaction" into an input form. This input is provided by the user in the form of text data.

[0801] Input: Text data about a child's disability or difficulties (e.g., "autism spectrum disorder" or "difficulty with social interaction")

[0802] Output: Structured data (JSON format)

[0803] Step 2:

[0804] The terminal converts the information entered by the user into structured data (JSON format), which organizes the data in a consistent format, and then transmits the converted data to the server using a secure communication protocol (HTTPS).

[0805] Input: Text data (information entered by the user)

[0806] Output: Structured data (JSON format), submitted data

[0807] Step 3:

[0808] The server receives the structured data sent from the terminal. The received data is first preprocessed, such as text normalization and tokenization, to remove unnecessary symbols and whitespace to ensure data consistency and integrity.

[0809] Input: Structured data (JSON format)

[0810] Output: Preprocessed data (text normalized, tokenized data)

[0811] Step 4:

[0812] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed data. The generative AI model uses machine learning algorithms to analyze the data and generate optimal solutions by referring to past case data. In this process, new support methods and practical examples are created based on the data entered by the user.

[0813] Input: Preprocessed data (text normalized, tokenized data)

[0814] Output: Support methods and practical examples

[0815] Step 5:

[0816] The server then organizes the generated support methods and practical examples in a structured data format (JSON format) and sends it to the user's device. At this stage, the data is formatted so that it can be displayed visually in an easy-to-understand manner.

[0817] Input: Support methods and practical examples

[0818] Output: Structured data (JSON format), submitted structured data

[0819] Step 6:

[0820] The device interprets the structured data received from the server and displays it in a format that is easy for the user to understand. Specifically, it visually presents specific support methods and practical examples along with titles such as "Communication Support Methods for Autism Spectrum Disorder."

[0821] Input: Structured data (JSON format), data received from the server

[0822] Output: A visually easy-to-understand display for the user

[0823] At each step, data processing and calculations are performed based on the input and output to ensure the smooth functioning of the entire system, and the system is designed to enable users to obtain assistance methods quickly and effectively.

[0824] (Application example 1)

[0825] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0826] In addition to providing appropriate assistance methods for individuals with disabilities, there is a need for a system that quickly provides efficient work procedures and problem-solving methods to factory workers. However, because these assistance methods and work procedures vary depending on individual circumstances, a one-size-fits-all solution is insufficient. Furthermore, delays in providing real-time assistance methods and work procedures can lead to reduced productivity and work efficiency.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0828] In this invention, the server includes: means for inputting information about a problem faced by an individual with a disability; means for transmitting the input information to the server; means for the server receiving the input information and searching for an optimal assistance method using a generative AI model; means for the server transmitting the assistance method generated using the generative AI model and practical examples to a terminal; means for the terminal displaying the transmitted assistance method and practical examples; means for the terminal inputting a factory work efficiency improvement method or problem-solving method and generating an optimal work procedure using a generative AI model; means for the server transmitting the generated work procedure and related examples to the terminal; and means for the terminal displaying the transmitted work procedure and related examples. This makes it possible to provide not only appropriate and personalized assistance methods for individuals with disabilities, but also efficient work procedures and problem-solving methods in real time to factory workers.

[0829] "Individual with a disability" means a person who has a physical or mental impairment.

[0830] "Information about problems" refers to data about specific challenges and problems that individuals face in their daily lives and at work.

[0831] "Server" refers to a computer system that receives, processes, and transmits information over a network.

[0832] A "generative AI model" refers to an artificial intelligence system that uses machine learning algorithms to automatically generate optimal solutions and procedures.

[0833] "Optimal support methods" refer to the most effective and appropriate responses and support methods for an individual's problems.

[0834] "Practical examples" refer to examples of support methods or solutions that have been shown to be effective based on specific situations or cases.

[0835] A "terminal" is a device that allows users to input and output data, and in this case refers to tablets and smart glasses used by factory workers.

[0836] "Improving factory work efficiency" refers to improving productivity by optimizing work procedures and the work environment in a factory.

[0837] A "problem-solving method" refers to a means or procedure for solving a specific problem or obstacle.

[0838] "Work procedures" refer to the specific steps and methods for performing work or operations within a factory.

[0839] "Relevant examples" refer to specific examples or case studies of successful implementations of similar situations or challenges.

[0840] This invention is a system that quickly provides appropriate support methods and efficiency procedures for individuals and factory workers with disabilities. It mainly consists of the following specific components:

[0841] 1. User Device:

[0842] This is a device for inputting information about the difficulties faced by individuals with disabilities and the efficiency of factory work. This includes devices such as tablets and smart glasses. For example, workers can input information about "pick-and-place operations."

[0843] 2. Server:

[0844] This is a computer system that receives and preprocesses information sent from the user's device. This server contains a generative AI model and automatically generates optimal support methods and work procedures based on the input information. Cloud-based servers such as AWS EC2 and Google Cloud Platform are often used.

[0845] 3. Generative AI Model:

[0846] These AI systems use machine learning algorithms to generate optimal solutions based on input information, such as OpenAI's GPT series. These AI models can provide personalized assistance for individuals with disabilities or optimal work procedures for factory workers.

[0847] Example

[0848] Improving factory work efficiency

[0849] Users input information about issues with "pick-and-place operations" and "frequent machine stoppages" via a tablet. Based on this, the server uses a generative AI model to generate optimal work procedures, such as "recalibrating sensors" and "introducing a high-speed vision system." The generated procedures and related cases are sent to the terminal, allowing workers to take immediate action.

[0850] For example, enter the following prompt:

[0851] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[0852] Hardware and software used

[0853] Hardware:

[0854] Smart glasses (e.g. Microsoft HoloLens)

[0855] Tablets (e.g. iPad Pro)

[0856] Factory robots (e.g. ABB IRB series, KUKA KR series)

[0857] software:

[0858] Cloud servers (e.g. AWS EC2, Google Cloud Platform)

[0859] Machine learning API (e.g. OpenAI GPT series)

[0860] Data processing and calculation

[0861] The server first preprocesses the information received from the user device. This preprocessing includes text normalization and tokenization. The generative AI model then generates optimal assistance methods and work procedures based on the preprocessed data. This generated data is then restructured and sent to the user device. The user device displays the received data in real time and converts it into a format that workers can use immediately.

[0862] Thus, the invention demonstrates how specific procedures, hardware, and software can be used to provide effective support methods and procedures for individuals and factory workers with disabilities.

[0863] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0864] Step 1:

[0865] A user uses a terminal to input information on how to improve the efficiency of factory work and how to solve problems.

[0866] Input: Work task (e.g., "pick-and-place operation") and associated problem (e.g., "frequent machine stoppages").

[0867] Users use tablets or smart glasses to input information about the specific task or problem they are facing into the interface.

[0868] Step 2:

[0869] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using HTTPS.

[0870] Input: Work tasks and associated issues entered by the user.

[0871] Data is transmitted securely, ensuring consistency and integrity of information.

[0872] Step 3:

[0873] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and inputs it into a generative AI model.

[0874] Input: Structured data (work tasks and associated problems).

[0875] Data preprocessing transforms data into a format that is easy for generative AI models to analyze.

[0876] Step 4:

[0877] The server's generative AI model generates optimal work procedures and problem-solving methods based on the pre-processed data.

[0878] Input: Preprocessed data.

[0879] A generative AI model (e.g., OpenAI GPT series) performs analysis and generates efficient work procedures (e.g., "recalibrate sensors" or "introduce high-speed vision systems") and related cases.

[0880] Step 5:

[0881] The server restructures the generated work procedures and related cases and sends them to the terminal in JSON format.

[0882] Input: Generated work procedures and related cases.

[0883] The information is organized in a structured data format and converted into a form that is easy for end users to use.

[0884] Step 6:

[0885] The terminal interprets the received data and displays it in a visually easy-to-understand format for the user.

[0886] Input: Structured data sent by the server.

[0887] Users can check optimal work procedures and related examples in real time through tablets or smart glasses and immediately apply them to their work.

[0888] As an example, use the following prompt:

[0889] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[0890] This series of steps enables efficient work support through collaboration between the server and the terminal. This system is capable of seamlessly inputting information, generating optimal solutions, and then implementing them.

[0891] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0892] This invention is a system that provides individualized support to children with disabilities, and aims to improve the quality of support by using a generative AI model to propose optimal support methods and combining it with an emotion engine that recognizes the user's emotions. This system is mainly composed of a user terminal, a server, a generative AI model, and an emotion engine.

[0893] System Configuration

[0894] The system of the present invention comprises the following elements:

[0895] 1. User terminal: A device that provides an interface for teachers to input information and display the results.

[0896] 2. Server: Receives information from the user device and drives the generative AI model and emotion engine to generate and provide the optimal support method.

[0897] 3. Generative AI model: A machine learning algorithm that generates optimal assistance methods based on user input information and emotional data.

[0898] 4. Emotion Engine: An analytical engine that recognizes user emotions and provides feedback to the generative AI model.

[0899] Program processing

[0900] The system of the present invention performs processing in the following procedure.

[0901] User Input

[0902] The user (teacher) uses the terminal to input information about the child's disability or difficulties, such as "autism spectrum disorder" or "difficulty with social interaction."

[0903] Sending data

[0904] The terminal converts the entered information into a structured data format and sends it to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data.

[0905] Server reception and analysis

[0906] The server analyzes and preprocesses the data received from the device, including normalizing and tokenizing the text. It also analyzes the user's voice and facial expressions to obtain emotional data.

[0907] Running the Emotion Engine

[0908] The emotion engine detects emotions from the user's voice and facial expressions and feeds that data back to the generative AI model, so that the user's emotional state is reflected in the process of generating assistance methods.

[0909] Running generative AI

[0910] The server's generative AI model generates optimal support methods and related practical examples based on the input data and emotional data. For example, it suggests specific support methods such as "communication practice using picture cards."

[0911] Processing and sending results

[0912] The server organizes the generated support methods and practical examples into a data package, converts it back into a structured data format, and sends it to the terminal. This process also uses a secure protocol.

[0913] Displaying the results

[0914] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0915] Specific examples

[0916] For example, if a user inputs information such as "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting, this emotional data is fed back to the generation AI. Taking the user's emotional state into consideration, the generation AI suggests "communication practice using picture cards" and even adds advice such as "it is recommended to practice in an environment where the user can relax." This information is presented to the user via their device, allowing for prompt and appropriate support.

[0917] In this way, by combining emotion engines, it becomes possible to provide a more appropriate support method that takes into account the user's emotional state, thereby significantly improving the quality of support.

[0918] The processing flow will be explained below.

[0919] Step 1:

[0920] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[0921] Step 2:

[0922] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent to the server.

[0923] Step 3:

[0924] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[0925] Step 4:

[0926] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[0927] Step 5:

[0928] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[0929] Step 6:

[0930] The user activates the emotion engine, which analyzes the user's voice and facial expressions in real time and generates emotion data.

[0931] Step 7:

[0932] The server receives emotion data from the emotion engine and integrates it with the pre-processed data, which is then fed into the generative AI model.

[0933] Step 8:

[0934] The server inputs the integrated data into a generative AI model to generate optimal support methods and practical examples, such as "communication practice using picture cards."

[0935] Step 9:

[0936] The server organizes the generated support methods and practice examples into a data package, which includes details of the support methods and practice examples.

[0937] Step 10:

[0938] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[0939] Step 11:

[0940] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[0941] Step 12:

[0942] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[0943] Example 2

[0944] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0945] When providing individualized support to children with disabilities, the effectiveness of the support can be reduced due to standardization of support methods or differences in implementation. Furthermore, the emotions and mental states of teachers and supporters affect the effectiveness of support methods, but conventional methods have had difficulty taking these into account. The present invention aims to solve these problems and improve the quality of support.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0947] In this invention, the server includes means for generating an optimal support method using a generative AI model, means for acquiring emotional data using an emotion engine that analyzes the user's emotional data and feeding the data back to the generative AI model, and means for transmitting the support method generated using the generative AI model and practical examples to a terminal, thereby making it possible to provide a more appropriate support method according to each individual situation and emotional state.

[0948] "Disability" refers to a condition in which a person has limitations in physical or mental function, and has difficulty in carrying out daily life or social activities.

[0949] "Input means" refers to a device or interface for receiving information in digital form, and typically includes a keyboard, mouse, tablet, or touchscreen.

[0950] "Server" refers to a high-performance computer used for large amounts of data processing and calculations, and provides services to other computers and devices over a network.

[0951] "Generative AI model" means a system that is programmed based on machine learning algorithms and uses new data to automatically generate optimal assistance methods.

[0952] "Emotion engine" refers to software or hardware that analyzes emotions from data such as a user's voice, facial expressions, and text, and identifies the user's emotional state.

[0953] "Feedback means" refers to a mechanism or method for implementing a process that returns or provides specific information to another system or device.

[0954] A "structured data format" is a format in which data is organized in an orderly manner according to certain rules, and generally includes formats such as JSON, XML, and CSV.

[0955] "Secure protocol" refers to a communication protocol for ensuring security in data communication, and generally includes HTTPS, SSL / TLS, etc.

[0956] "Terminal" refers to an electronic device that is directly operated by a user, including, for example, a PC, tablet, or smartphone.

[0957] "Supportive measures" refers to the specific methods and procedures that help individuals with disabilities to carry out daily, academic, and social activities.

[0958] "Practical cases" refer to data showing records and results of specific support activities carried out in the past, which other users can use as reference.

[0959] "Preprocessing methods" refers to the processes and techniques used to transform raw data into a form suitable for analysis, including data cleaning, normalization, and tokenization.

[0960] This invention is a system that provides individualized support for children with disabilities. This system mainly consists of a user terminal, a server, a generative AI model, and an emotion engine.

[0961] Role of user terminal

[0962] The user (teacher) operates a device to input information about the child's disability or difficulties. The device is typically a PC or tablet. Input is done in form format, and specific information includes "autism spectrum disorder" and "difficulties with social interaction." After input, the device converts this information into a structured data format (e.g., JSON format) and securely sends it to the server via the HTTPS protocol.

[0963] Server Reception and Preprocessing

[0964] The server receives information sent from the user's device. After receiving the data, the server performs preprocessing such as text normalization and tokenization, and then analyzes the data. Preprocessing steps include removing unnecessary spaces and special characters, and segmenting words. The server also simultaneously collects voice and facial expression data provided by the user, and analyzes the emotional data using an emotion engine.

[0965] Running emotion engines and generative AI models

[0966] The server uses an emotion engine to identify the user's emotional state from their voice and facial expression data. This makes it possible to accurately detect, for example, whether the user is feeling stressed. This emotion data is fed back to the generative AI model. Based on the received input data and emotion data, the generative AI model generates optimal support methods and practical examples. As a specific example, it suggests a support method such as "communication practice using picture cards," and also adds the advice that "it is recommended to practice in a relaxed environment."

[0967] Sending and displaying results

[0968] The server organizes the generated support methods and practical examples and converts them back into structured data format. This data package is then securely sent to the user's device via the HTTPS protocol. The device retrieves the received data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen.

[0969] Examples and prompts

[0970] As a concrete example, consider a case where a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting. This emotion data is fed back to the generative AI model, which then suggests "communication practice using picture cards." It also adds the advice that "it is recommended to practice in a relaxing environment." This information is displayed on the device, allowing the user to provide prompt and appropriate support.

[0971] Prompt Sentence Examples

[0972] The input prompt for the generative AI model is:

[0973] "Disability: Autism Spectrum Disorder"

[0974] "Difficulties: Difficulties in social interactions"

[0975] "Emotion: Stress, Intensity: High"

[0976] The generative AI model then proposes optimal assistance methods based on these prompts, which can be directly input into the system and reflect the user's specific concerns and emotional state.

[0977] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0978] Program processing flow

[0979] Step 1:

[0980] User Input

[0981] The user (teacher) operates the device to input information about the child's disability or difficulties. The input method is a form format, and specific information such as "autism spectrum disorder" or "difficulty in social interaction" is entered into the device. The data obtained from this input becomes the initial input data for the system.

[0982] Step 2:

[0983] Sending data

[0984] The device converts the input information into a structured data format such as JSON, for example, "{\"Disability\": \"Autism Spectrum Disorder\", \"Problem\": \"Difficulty with Social Interaction\"}". The converted data is then sent to the server using the HTTPS protocol. Data integrity and confidentiality are ensured at this stage.

[0985] Step 3:

[0986] Server Reception and Preprocessing

[0987] The server receives the structured data sent from the device. It then performs preprocessing, normalization and tokenization, to remove unnecessary whitespace and special characters from the received data and to separate words. For example, the text "autism spectrum disorder" is divided into tokens such as "autism," "spectrum," and "disorder."

[0988] Step 4:

[0989] Acquiring user emotion data

[0990] The server receives the voice and facial expression data provided by the user at the same time. The server inputs this data into an emotion engine to analyze the user's emotional state. This analysis includes the tone and speaking rate of the voice data, and facial expression analysis of the facial expression data. For example, if the voice analysis detects high levels of "stress," this information is captured as emotion data.

[0991] Step 5:

[0992] Emotional Data Feedback

[0993] The server feeds back the acquired emotional data (e.g., "Emotion: Stress, Intensity: High") to the generative AI model. This feedback process allows the generative AI model to reflect the user's emotional state in the generation process of the assistance method.

[0994] Step 6:

[0995] Running generative AI models

[0996] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed input data and the fed-back emotional data. Using machine learning algorithms, the generative AI model suggests, for example, "communication practice using picture cards." It also provides advice such as "It is recommended to practice in an environment where the user can relax."

[0997] Step 7:

[0998] Processing and sending results

[0999] The server organizes the generated support methods and practice examples and converts them back into structured data in JSON format. The converted data package (e.g., "{\"Support method\": \"Communication practice using picture cards\", \"Recommended situation\": \"Perform in a relaxed environment\"}") is sent to the device using the HTTPS protocol.

[1000] Step 8:

[1001] Displaying the results

[1002] The device retrieves the data received from the server and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen. Based on this information, the user can quickly implement specific support methods.

[1003] (Application example 2)

[1004] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1005] Autonomous vehicles lack the ability to properly understand the emotional state of pedestrians and provide appropriate assistance based on that understanding. As a result, pedestrian safety and a sense of security cannot be fully ensured. Furthermore, conventional autonomous driving systems have difficulty adapting flexibly to the surrounding situation, and communication with pedestrians in particular is lacking.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1007] In this invention, the server includes means for inputting information about the difficulties faced by an individual with a disability, means for transmitting the input information to the server, means for the server to receive the input information and search for an optimal assistance method using a generative AI model and an emotion engine, means for the server to transmit the assistance method and practical example generated using the generative AI model and emotion engine to a terminal, means for the terminal to display the transmitted assistance method and practical example, means for the system to collect video and audio data from a camera and microphone mounted on a mobile vehicle and analyze the emotional state from the data, and means for providing appropriate assistance methods to nearby users based on the emotional data analyzed by the generative AI model. This enables flexible and appropriate assistance to ensure safety and security by appropriately understanding the emotional state of pedestrians.

[1008] "Information on the challenges faced by individuals with disabilities" is data that documents the specific problems and difficulties faced by individuals with disabilities.

[1009] The "server" is a high-performance computer that processes data received from user terminals via the network and generates and distributes the necessary support methods.

[1010] A "generative AI model" is a machine learning algorithm that suggests optimal support methods based on information input by the user.

[1011] The "emotion engine" is a program that analyzes the user's emotions from video and audio data and provides appropriate feedback to the generative AI model.

[1012] A "terminal" is an electronic device that allows a user to input information and receive and display the generated support methods and practical examples.

[1013] "Mobile object" refers to a moving mechanical device such as an autonomous vehicle, equipped with sensors to sense its surroundings.

[1014] A "camera" is a photographic device used to collect video data and record the facial expressions and movements of pedestrians.

[1015] A "microphone" is a device for collecting audio data and is used to record surrounding sounds and speech.

[1016] "Video data" refers to visual information collected by a camera in the form of images or videos that are subject to analysis.

[1017] "Audio Data" means audio information collected by a microphone in the form of sound or speech that is subject to analysis.

[1018] The "means for analyzing emotional state" refers to a program or algorithm for determining the user's emotions based on video and audio data.

[1019] The "means for providing an appropriate support method" is a program or device for providing the user with the most appropriate response based on the analyzed emotional state.

[1020] A "structured data format" is a format for arranging data into a unified and easy-to-use format.

[1021] The specific system configuration and operation procedure for an embodiment of the present invention are described below. The system is broadly composed of a user terminal, a server, a generative AI model, an emotion engine, and a mobile object (an autonomously driven vehicle).

[1022] User terminal configuration and functions

[1023] A user terminal is an electronic device for inputting information about the difficulties faced by individuals with disabilities. The terminal provides an information input interface that allows users to input the type of disability and specific difficulties. Specifically, the terminal has an interface for typing information such as autism spectrum disorder and difficulties with social interaction.

[1024] Server configuration and functions

[1025] The server receives the information sent from the user device, analyzes the data, and executes the generative AI model. The server uses the following software in particular:

[1026] TensorFlow: Running Generative AI Models

[1027] OpenCV: Video data preprocessing and analysis

[1028] EmotionAPI: Emotion Engine

[1029] The server analyzes the data sent from the user's device and generates optimal support methods and related practical examples using a generative AI model. Furthermore, it appropriately adjusts the generated support methods based on the emotional data analyzed by the emotion engine.

[1030] Linking generative AI models and emotion engines

[1031] The generative AI model is a machine learning algorithm that generates assistance methods based on information and emotional data input by the user. During this process, the emotion engine analyzes the user's emotional state and provides feedback to the generative AI model to provide more appropriate assistance methods.

[1032] Mobile device configuration and functions

[1033] This system uses cameras and microphones mounted on moving objects (autonomous vehicles) to collect video and audio data in real time. The emotion engine analyzes this data and determines the emotional state of pedestrians. The generative AI model provides appropriate assistance methods to nearby users based on the analyzed emotional data.

[1034] As a concrete example, consider a situation where a pedestrian stands at a crosswalk while an autonomous vehicle is traveling down the road. At this time, the system captures the pedestrian's facial expressions with a camera, and the emotion engine analyzes whether the pedestrian is feeling "tension" or "stress." Based on this emotional data, the generative AI model suggests "stopping the vehicle" and notifies the driver via a display and audio alert saying, "Please be careful, pedestrians have the right of way." This allows the pedestrian to cross the road safely.

[1035] Prompt Sentence Examples

[1036] Based on the input data from the camera and microphone installed in the autonomous vehicle, determine whether the pedestrian is feeling stressed and generate the optimal response method based on the result. For example, if the pedestrian is feeling stressed, consider a response such as "the vehicle stopping and encouraging the pedestrian to cross first."

[1037] As a result, the present invention enables autonomous vehicles to flexibly adapt to their surroundings, ensuring safety and a sense of security for pedestrians.

[1038] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1039] Step 1: Collect data

[1040] Cameras and microphones installed in autonomous vehicles collect video and audio data of the surrounding area in real time. The cameras capture pedestrians' faces and gestures, and the microphones record environmental sounds and pedestrians' voices. The collected video and audio data become input to the system.

[1041] Step 2: Preprocessing the data

[1042] The server uses OpenCV to identify faces and facial expressions from the collected video data. Specifically, the video is converted to grayscale and a face detection algorithm is applied. Meanwhile, the audio data is denoised and normalised to extract acoustic features. This pre-processing results in clean data for analysis.

[1043] Step 3: Sentiment Analysis

[1044] The server uses the Emotion API to analyze the pedestrian's emotional state from preprocessed video and audio data. The preprocessed video and audio are used as input, and the pedestrian's emotional state (e.g., tension, stress, relief) is returned as output. Specifically, emotions are determined from subtle changes in facial expressions and voice tone.

[1045] Step 4: Generating assistance methods using generative AI models

[1046] The server inputs the analyzed emotion data into a generative AI model using TensorFlow. This model generates the optimal assistance method by taking into account the pedestrian's emotion. The input is the pedestrian's emotion data, and the output is the appropriate countermeasure (e.g., stopping the vehicle, audio notification).

[1047] Step 5: Communicate how to help

[1048] The server sends the generated assistance method to the display and voice system of the autonomous vehicle. The display will show a message saying "Please be careful, pedestrians have the right of way," and the voice system will make a similar announcement. This allows the pedestrian to receive immediate and appropriate assistance.

[1049] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1050] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1051] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1052] [Fourth embodiment]

[1053] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1054] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1055] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1056] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1057] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1058] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1059] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1060] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1061] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1062] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1063] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1064] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1065] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1066] This invention is a system that supports individual support for children with disabilities. In particular, the system allows users (teachers) to input information about disabilities and difficulties, and uses a generative AI model to search for optimal support methods and practical examples, displaying the results on a terminal to provide fast and effective support.

[1067] System Configuration

[1068] This system mainly consists of the following components:

[1069] 1. User terminal: A device used by the teacher that provides an interface for entering information and displaying results.

[1070] 2. Server: Receives information from the user device, executes the generative AI model, and generates appropriate support methods and practical examples.

[1071] 3. Generative AI model: A machine learning algorithm that searches for and generates optimal support methods based on information about disabilities and difficulties.

[1072] Program processing

[1073] The program process for this system is as follows:

[1074] User Input

[1075] The user (teacher) uses the device interface to input information about the child's disability or difficulties, such as information about autism spectrum disorder or difficulties with social interaction.

[1076] Sending data

[1077] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using a secure protocol (HTTPS). At this stage, the consistency and integrity of the data is ensured.

[1078] Server reception and analysis

[1079] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and feeds it into a generative AI model. This preprocessing transforms the data into a format that is easier to analyze.

[1080] Running generative AI

[1081] The server's generative AI model searches for the optimal support method based on the input data and generates related practical examples, such as "communication support using picture cards."

[1082] Processing and sending results

[1083] The server then reorganizes the generated support methods and practical examples in a structured data format and sends it to the terminal. At this stage, the data is organized in a form that can be used immediately by the user.

[1084] Displaying the results

[1085] The device interprets the received data and displays it in a format that is easy for the user to understand. For example, it may visually present something like "Communication support methods for autism spectrum disorders" on the screen.

[1086] Specific examples

[1087] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," this information is sent to the server. The server uses generative AI to generate a result that "communication practice using picture cards" is effective, and sends this result along with appropriate examples to the device. As a result, the user can quickly obtain specific support methods and apply them in actual support.

[1088] As described above, the present invention specifically shows an embodiment for quickly and effectively providing an appropriate support method for children with disabilities.

[1089] The processing flow will be explained below.

[1090] Step 1:

[1091] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[1092] Step 2:

[1093] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent.

[1094] Step 3:

[1095] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[1096] Step 4:

[1097] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[1098] Step 5:

[1099] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[1100] Step 6:

[1101] The server inputs the preprocessed data into a generative AI model, which generates optimal support methods and practical examples based on the input data. For example, it outputs a specific support method such as "communication practice using picture cards."

[1102] Step 7:

[1103] The server organizes the output from the generative AI model into a data package, which includes details of assistance methods and practical examples.

[1104] Step 8:

[1105] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[1106] Step 9:

[1107] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[1108] Step 10:

[1109] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[1110] Example 1

[1111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1112] It is difficult to find a method for providing individualized support to children with disabilities quickly and effectively using conventional methods. In particular, they lack the ability to properly analyze input information and generate specific and practical support methods. There is also a need to display the generated information in a format that is easy for users to understand. The present invention aims to provide a system that solves these problems.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1114] In this invention, the server includes means for receiving input information and performing preprocessing such as text normalization and tokenization, means for generating optimal support methods and practical examples from the preprocessed information using a generative AI model, and means for transmitting the generated support methods and practical examples to the terminal in structured data format, thereby making it possible to quickly and effectively analyze the input information and provide specific and practical support methods to the user.

[1115] "Disability and difficulty information" is data that details the specific circumstances, problems, and difficulties of individuals with disabilities.

[1116] "Structured data" is a data format that is organized based on a specific format or standard, and is usually expressed in formats such as JSON or XML.

[1117] A "secure communication protocol" is a communication technology that prevents data exchange from being eavesdropped on or tampered with by third parties, and includes HTTPS.

[1118] "Preprocessing" refers to processes such as text normalization and tokenization to make data easier for generative AI models to parse.

[1119] "Normalization" refers to the process of organizing text data by removing unnecessary symbols and spaces.

[1120] "Tokenization" refers to the process of dividing text data into units such as words and phrases.

[1121] A "generative AI model" is a model that uses machine learning algorithms to generate optimal support methods and practical examples based on input data.

[1122] "Assistive techniques" refers to specific aids or approaches provided to individuals with disabilities.

[1123] "Practical cases" are data that describe specific successful cases and methods of support that have been provided in the past.

[1124] A "structured data format" is a form of data that is systematically organized in a specific format, commonly including JSON and XML.

[1125] A "terminal" is a device, such as a computer, tablet, or smartphone, that a user uses to enter information and view results.

[1126] The "format that is easy for users to understand" is a format in which the generated support methods and practical examples are displayed in a visually easy-to-understand manner so that the user can easily understand them.

[1127] This invention is a system that provides individualized support to children with disabilities quickly and effectively. This system mainly consists of a user terminal, a server, and a generative AI model.

[1128] User terminal

[1129] User terminals are devices used by teachers and supporters, and provide an interface for inputting information about children's disabilities and difficulties. Specifically, PCs, tablets, smartphones, etc. are used. The terminals are equipped with a dedicated form where users can enter various information, and the information entered through this form is converted into structured data (e.g., JSON format). This structured data is sent to the server using a secure communication protocol such as HTTPS.

[1130] server

[1131] The server receives the information sent from the user device and performs preprocessing on the data. Specific preprocessing includes normalizing and tokenizing the text. For example, it removes unnecessary symbols and spaces from the text and checks the integrity of the data. This preprocessing prepares the data in a format that is easy for the generative AI model to analyze.

[1132] The server uses a generative AI model to generate optimal support methods and practical examples from the preprocessed data. The generated support methods and practical examples are then organized again in a structured data format and sent to the user's device. At this time, the generated information is prepared to be displayed in a visually easy-to-understand format.

[1133] Generative AI Models

[1134] The generative AI model uses machine learning algorithms to generate optimal support methods and practical examples based on input data. This server-based model analyzes detailed information about disabilities and problems, and suggests optimal support methods based on past support data.

[1135] Specific examples

[1136] For example, if a user inputs the information "autism spectrum disorder" and "difficulty with social interaction" on a device, this information is converted into JSON format and sent to the server using the secure HTTPS protocol. The server receives this information and normalizes and tokenizes the text. The generative AI model then analyzes this data and generates a "communication support method using picture cards" and specific practical examples. This generated information is then sent back to the device in structured data format and displayed in a format that is easy for the user to understand.

[1137] Prompt Sentence Examples

[1138] Examples of prompts that users can input to a generative AI model include:

[1139] "When a user inputs the information 'autism spectrum disorder' and 'difficulty in social interaction,' please generate the optimal assistance method."

[1140] Thus, the present invention specifically illustrates an embodiment for providing individualized support to children with disabilities quickly and effectively.

[1141] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1142] Step 1:

[1143] The user (teacher) uses the terminal interface to input information about the child's disability or difficulties. Specifically, the user enters information such as "autism spectrum disorder" or "difficulty in social interaction" into an input form. This input is provided by the user in the form of text data.

[1144] Input: Text data about a child's disability or difficulties (e.g., "autism spectrum disorder" or "difficulty with social interaction")

[1145] Output: Structured data (JSON format)

[1146] Step 2:

[1147] The terminal converts the information entered by the user into structured data (JSON format), which organizes the data in a consistent format, and then transmits the converted data to the server using a secure communication protocol (HTTPS).

[1148] Input: Text data (information entered by the user)

[1149] Output: Structured data (JSON format), submitted data

[1150] Step 3:

[1151] The server receives the structured data sent from the terminal. The received data is first preprocessed, such as text normalization and tokenization, to remove unnecessary symbols and whitespace to ensure data consistency and integrity.

[1152] Input: Structured data (JSON format)

[1153] Output: Preprocessed data (text normalized, tokenized data)

[1154] Step 4:

[1155] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed data. The generative AI model uses machine learning algorithms to analyze the data and generate optimal solutions by referring to past case data. In this process, new support methods and practical examples are created based on the data entered by the user.

[1156] Input: Preprocessed data (text normalized, tokenized data)

[1157] Output: Support methods and practical examples

[1158] Step 5:

[1159] The server then organizes the generated support methods and practical examples in a structured data format (JSON format) and sends it to the user's device. At this stage, the data is formatted so that it can be displayed visually in an easy-to-understand manner.

[1160] Input: Support methods and practical examples

[1161] Output: Structured data (JSON format), submitted structured data

[1162] Step 6:

[1163] The device interprets the structured data received from the server and displays it in a format that is easy for the user to understand. Specifically, it visually presents specific support methods and practical examples along with titles such as "Communication Support Methods for Autism Spectrum Disorder."

[1164] Input: Structured data (JSON format), data received from the server

[1165] Output: A visually easy-to-understand display for the user

[1166] At each step, data processing and calculations are performed based on the input and output to ensure the smooth functioning of the entire system, and the system is designed to enable users to obtain assistance methods quickly and effectively.

[1167] (Application example 1)

[1168] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1169] In addition to providing appropriate assistance methods for individuals with disabilities, there is a need for a system that quickly provides efficient work procedures and problem-solving methods to factory workers. However, because these assistance methods and work procedures vary depending on individual circumstances, a one-size-fits-all solution is insufficient. Furthermore, delays in providing real-time assistance methods and work procedures can lead to reduced productivity and work efficiency.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1171] In this invention, the server includes: means for inputting information about a problem faced by an individual with a disability; means for transmitting the input information to the server; means for the server receiving the input information and searching for an optimal assistance method using a generative AI model; means for the server transmitting the assistance method generated using the generative AI model and practical examples to a terminal; means for the terminal displaying the transmitted assistance method and practical examples; means for the terminal inputting a factory work efficiency improvement method or problem-solving method and generating an optimal work procedure using a generative AI model; means for the server transmitting the generated work procedure and related examples to the terminal; and means for the terminal displaying the transmitted work procedure and related examples. This makes it possible to provide not only appropriate and personalized assistance methods for individuals with disabilities, but also efficient work procedures and problem-solving methods in real time to factory workers.

[1172] "Individual with a disability" means a person who has a physical or mental impairment.

[1173] "Information about problems" refers to data about specific challenges and problems that individuals face in their daily lives and at work.

[1174] "Server" refers to a computer system that receives, processes, and transmits information over a network.

[1175] A "generative AI model" refers to an artificial intelligence system that uses machine learning algorithms to automatically generate optimal solutions and procedures.

[1176] "Optimal support methods" refer to the most effective and appropriate responses and support methods for an individual's problems.

[1177] "Practical examples" refer to examples of support methods or solutions that have been shown to be effective based on specific situations or cases.

[1178] A "terminal" is a device that allows users to input and output data, and in this case refers to tablets and smart glasses used by factory workers.

[1179] "Improving factory work efficiency" refers to improving productivity by optimizing work procedures and the work environment in a factory.

[1180] A "problem-solving method" refers to a means or procedure for solving a specific problem or obstacle.

[1181] "Work procedures" refer to the specific steps and methods for performing work or operations within a factory.

[1182] "Relevant examples" refer to specific examples or case studies of successful implementations of similar situations or challenges.

[1183] This invention is a system that quickly provides appropriate support methods and efficiency procedures for individuals and factory workers with disabilities. It mainly consists of the following specific components:

[1184] 1. User Device:

[1185] This is a device for inputting information about the difficulties faced by individuals with disabilities and the efficiency of factory work. This includes devices such as tablets and smart glasses. For example, workers can input information about "pick-and-place operations."

[1186] 2. Server:

[1187] This is a computer system that receives and preprocesses information sent from the user's device. This server contains a generative AI model and automatically generates optimal support methods and work procedures based on the input information. Cloud-based servers such as AWS EC2 and Google Cloud Platform are often used.

[1188] 3. Generative AI Model:

[1189] These AI systems use machine learning algorithms to generate optimal solutions based on input information, such as OpenAI's GPT series. These AI models can provide personalized assistance for individuals with disabilities or optimal work procedures for factory workers.

[1190] Example

[1191] Improving factory work efficiency

[1192] Users input information about issues with "pick-and-place operations" and "frequent machine stoppages" via a tablet. Based on this, the server uses a generative AI model to generate optimal work procedures, such as "recalibrating sensors" and "introducing a high-speed vision system." The generated procedures and related cases are sent to the terminal, allowing workers to take immediate action.

[1193] For example, enter the following prompt:

[1194] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[1195] Hardware and software used

[1196] Hardware:

[1197] Smart glasses (e.g. Microsoft HoloLens)

[1198] Tablets (e.g. iPad Pro)

[1199] Factory robots (e.g. ABB IRB series, KUKA KR series)

[1200] software:

[1201] Cloud servers (e.g. AWS EC2, Google Cloud Platform)

[1202] Machine learning API (e.g. OpenAI GPT series)

[1203] Data processing and calculation

[1204] The server first preprocesses the information received from the user device. This preprocessing includes text normalization and tokenization. The generative AI model then generates optimal assistance methods and work procedures based on the preprocessed data. This generated data is then restructured and sent to the user device. The user device displays the received data in real time and converts it into a format that workers can use immediately.

[1205] Thus, the invention demonstrates how specific procedures, hardware, and software can be used to provide effective support methods and procedures for individuals and factory workers with disabilities.

[1206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1207] Step 1:

[1208] A user uses a terminal to input information on how to improve the efficiency of factory work and how to solve problems.

[1209] Input: Work task (e.g., "pick-and-place operation") and associated problem (e.g., "frequent machine stoppages").

[1210] Users use tablets or smart glasses to input information about the specific task or problem they are facing into the interface.

[1211] Step 2:

[1212] The terminal converts the input information into structured data (e.g., JSON) and sends it to the server using HTTPS.

[1213] Input: Work tasks and associated issues entered by the user.

[1214] Data is transmitted securely, ensuring consistency and integrity of information.

[1215] Step 3:

[1216] The server preprocesses the received data (e.g., normalizes and tokenizes the text) and inputs it into a generative AI model.

[1217] Input: Structured data (work tasks and associated problems).

[1218] Data preprocessing transforms data into a format that is easy for generative AI models to analyze.

[1219] Step 4:

[1220] The server's generative AI model generates optimal work procedures and problem-solving methods based on the pre-processed data.

[1221] Input: Preprocessed data.

[1222] A generative AI model (e.g., OpenAI GPT series) performs analysis and generates efficient work procedures (e.g., "recalibrate sensors" or "introduce high-speed vision systems") and related cases.

[1223] Step 5:

[1224] The server restructures the generated work procedures and related cases and sends them to the terminal in JSON format.

[1225] Input: Generated work procedures and related cases.

[1226] The information is organized in a structured data format and converted into a form that is easy for end users to use.

[1227] Step 6:

[1228] The terminal interprets the received data and displays it in a visually easy-to-understand format for the user.

[1229] Input: Structured data sent by the server.

[1230] Users can check optimal work procedures and related examples in real time through tablets or smart glasses and immediately apply them to their work.

[1231] As an example, use the following prompt:

[1232] "Please suggest the best work procedures and solutions to the problems of 'frequent machine stoppages' and 'inaccurate item placement' in 'pick and place operations' in the factory."

[1233] This series of steps enables efficient work support through collaboration between the server and the terminal. This system is capable of seamlessly inputting information, generating optimal solutions, and then implementing them.

[1234] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1235] This invention is a system that provides individualized support to children with disabilities, and aims to improve the quality of support by using a generative AI model to propose optimal support methods and combining it with an emotion engine that recognizes the user's emotions. This system is mainly composed of a user terminal, a server, a generative AI model, and an emotion engine.

[1236] System Configuration

[1237] The system of the present invention comprises the following elements:

[1238] 1. User terminal: A device that provides an interface for teachers to input information and display the results.

[1239] 2. Server: Receives information from the user device and drives the generative AI model and emotion engine to generate and provide the optimal support method.

[1240] 3. Generative AI model: A machine learning algorithm that generates optimal assistance methods based on user input information and emotional data.

[1241] 4. Emotion Engine: An analytical engine that recognizes user emotions and provides feedback to the generative AI model.

[1242] Program processing

[1243] The system of the present invention performs processing in the following procedure.

[1244] User Input

[1245] The user (teacher) uses the terminal to input information about the child's disability or difficulties, such as "autism spectrum disorder" or "difficulty with social interaction."

[1246] Sending data

[1247] The terminal converts the entered information into a structured data format and sends it to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data.

[1248] Server reception and analysis

[1249] The server analyzes and preprocesses the data received from the device, including normalizing and tokenizing the text. It also analyzes the user's voice and facial expressions to obtain emotional data.

[1250] Running the Emotion Engine

[1251] The emotion engine detects emotions from the user's voice and facial expressions and feeds that data back to the generative AI model, so that the user's emotional state is reflected in the process of generating assistance methods.

[1252] Running generative AI

[1253] The server's generative AI model generates optimal support methods and related practical examples based on the input data and emotional data. For example, it suggests specific support methods such as "communication practice using picture cards."

[1254] Processing and sending results

[1255] The server organizes the generated support methods and practical examples into a data package, converts it back into a structured data format, and sends it to the terminal. This process also uses a secure protocol.

[1256] Displaying the results

[1257] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[1258] Specific examples

[1259] For example, if a user inputs information such as "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting, this emotional data is fed back to the generation AI. Taking the user's emotional state into consideration, the generation AI suggests "communication practice using picture cards" and even adds advice such as "it is recommended to practice in an environment where the user can relax." This information is presented to the user via their device, allowing for prompt and appropriate support.

[1260] In this way, by combining emotion engines, it becomes possible to provide a more appropriate support method that takes into account the user's emotional state, thereby significantly improving the quality of support.

[1261] The processing flow will be explained below.

[1262] Step 1:

[1263] The user enters information about the child with a disability, for example, entering text such as "autism spectrum disorder" and "difficulty with social interaction" into the input fields.

[1264] Step 2:

[1265] The terminal converts the input information into a structured data format (e.g., JSON), and the converted data is prepared and ready to be sent to the server.

[1266] Step 3:

[1267] The terminal then securely transmits the converted data to the server using the HTTPS protocol, ensuring the integrity and confidentiality of the data during transmission.

[1268] Step 4:

[1269] The server receives the data from the device. The received data is received in JSON format, so it can be analyzed as is.

[1270] Step 5:

[1271] The server preprocesses the data it receives, which includes text normalization and tokenization, converting the data into a format that is easy for the generative AI model to process.

[1272] Step 6:

[1273] The user activates the emotion engine, which analyzes the user's voice and facial expressions in real time and generates emotion data.

[1274] Step 7:

[1275] The server receives emotion data from the emotion engine and integrates it with the pre-processed data, which is then fed into the generative AI model.

[1276] Step 8:

[1277] The server inputs the integrated data into a generative AI model to generate optimal support methods and practical examples, such as "communication practice using picture cards."

[1278] Step 9:

[1279] The server organizes the generated support methods and practice examples into a data package, which includes details of the support methods and practice examples.

[1280] Step 10:

[1281] The server then converts the organized data package back into a structured data format and sends it to the device, again using a secure protocol.

[1282] Step 11:

[1283] The device receives the data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method for autism spectrum disorder: Use of picture cards. Practical example: Successful example at XX school" may be displayed on the screen.

[1284] Step 12:

[1285] Users can plan and implement actual support activities based on the displayed information, making it possible to provide appropriate and individualized support to children.

[1286] Example 2

[1287] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] When providing individualized support to children with disabilities, the effectiveness of the support can be reduced due to standardization of support methods or differences in implementation. Furthermore, the emotions and mental states of teachers and supporters affect the effectiveness of support methods, but conventional methods have had difficulty taking these into account. The present invention aims to solve these problems and improve the quality of support.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1290] In this invention, the server includes means for generating an optimal support method using a generative AI model, means for acquiring emotional data using an emotion engine that analyzes the user's emotional data and feeding the data back to the generative AI model, and means for transmitting the support method generated using the generative AI model and practical examples to a terminal, thereby making it possible to provide a more appropriate support method according to each individual situation and emotional state.

[1291] "Disability" refers to a condition in which a person has limitations in physical or mental function, and has difficulty in carrying out daily life or social activities.

[1292] "Input means" refers to a device or interface for receiving information in digital form, and typically includes a keyboard, mouse, tablet, or touchscreen.

[1293] "Server" refers to a high-performance computer used for large amounts of data processing and calculations, and provides services to other computers and devices over a network.

[1294] "Generative AI model" means a system that is programmed based on machine learning algorithms and uses new data to automatically generate optimal assistance methods.

[1295] "Emotion engine" refers to software or hardware that analyzes emotions from data such as a user's voice, facial expressions, and text, and identifies the user's emotional state.

[1296] "Feedback means" refers to a mechanism or method for implementing a process that returns or provides specific information to another system or device.

[1297] A "structured data format" is a format in which data is organized in an orderly manner according to certain rules, and generally includes formats such as JSON, XML, and CSV.

[1298] "Secure protocol" refers to a communication protocol for ensuring security in data communication, and generally includes HTTPS, SSL / TLS, etc.

[1299] "Terminal" refers to an electronic device that is directly operated by a user, including, for example, a PC, tablet, or smartphone.

[1300] "Supportive measures" refers to the specific methods and procedures that help individuals with disabilities to carry out daily, academic, and social activities.

[1301] "Practical cases" refer to data showing records and results of specific support activities carried out in the past, which other users can use as reference.

[1302] "Preprocessing methods" refers to the processes and techniques used to transform raw data into a form suitable for analysis, including data cleaning, normalization, and tokenization.

[1303] This invention is a system that provides individualized support for children with disabilities. This system mainly consists of a user terminal, a server, a generative AI model, and an emotion engine.

[1304] Role of user terminal

[1305] The user (teacher) operates a device to input information about the child's disability or difficulties. The device is typically a PC or tablet. Input is done in form format, and specific information includes "autism spectrum disorder" and "difficulties with social interaction." After input, the device converts this information into a structured data format (e.g., JSON format) and securely sends it to the server via the HTTPS protocol.

[1306] Server Reception and Preprocessing

[1307] The server receives information sent from the user's device. After receiving the data, the server performs preprocessing such as text normalization and tokenization, and then analyzes the data. Preprocessing steps include removing unnecessary spaces and special characters, and segmenting words. The server also simultaneously collects voice and facial expression data provided by the user, and analyzes the emotional data using an emotion engine.

[1308] Running emotion engines and generative AI models

[1309] The server uses an emotion engine to identify the user's emotional state from their voice and facial expression data. This makes it possible to accurately detect, for example, whether the user is feeling stressed. This emotion data is fed back to the generative AI model. Based on the received input data and emotion data, the generative AI model generates optimal support methods and practical examples. As a specific example, it suggests a support method such as "communication practice using picture cards," and also adds the advice that "it is recommended to practice in a relaxed environment."

[1310] Sending and displaying results

[1311] The server organizes the generated support methods and practical examples and converts them back into structured data format. This data package is then securely sent to the user's device via the HTTPS protocol. The device retrieves the received data and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen.

[1312] Examples and prompts

[1313] As a concrete example, consider a case where a user inputs the information "autism spectrum disorder" and "difficulty with social interaction," and the emotion engine detects that the user is feeling stressed while inputting. This emotion data is fed back to the generative AI model, which then suggests "communication practice using picture cards." It also adds the advice that "it is recommended to practice in a relaxing environment." This information is displayed on the device, allowing the user to provide prompt and appropriate support.

[1314] Prompt Sentence Examples

[1315] The input prompt for the generative AI model is:

[1316] "Disability: Autism Spectrum Disorder"

[1317] "Difficulties: Difficulties in social interactions"

[1318] "Emotion: Stress, Intensity: High"

[1319] The generative AI model then proposes optimal assistance methods based on these prompts, which can be directly input into the system and reflect the user's specific concerns and emotional state.

[1320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1321] Program processing flow

[1322] Step 1:

[1323] User Input

[1324] The user (teacher) operates the device to input information about the child's disability or difficulties. The input method is a form format, and specific information such as "autism spectrum disorder" or "difficulty in social interaction" is entered into the device. The data obtained from this input becomes the initial input data for the system.

[1325] Step 2:

[1326] Sending data

[1327] The device converts the input information into a structured data format such as JSON, for example, "{\"Disability\": \"Autism Spectrum Disorder\", \"Problem\": \"Difficulty with Social Interaction\"}". The converted data is then sent to the server using the HTTPS protocol. Data integrity and confidentiality are ensured at this stage.

[1328] Step 3:

[1329] Server Reception and Preprocessing

[1330] The server receives the structured data sent from the device. It then performs preprocessing, normalization and tokenization, to remove unnecessary whitespace and special characters from the received data and to separate words. For example, the text "autism spectrum disorder" is divided into tokens such as "autism," "spectrum," and "disorder."

[1331] Step 4:

[1332] Acquiring user emotion data

[1333] The server receives the voice and facial expression data provided by the user at the same time. The server inputs this data into an emotion engine to analyze the user's emotional state. This analysis includes the tone and speaking rate of the voice data, and facial expression analysis of the facial expression data. For example, if the voice analysis detects high levels of "stress," this information is captured as emotion data.

[1334] Step 5:

[1335] Emotional Data Feedback

[1336] The server feeds back the acquired emotional data (e.g., "Emotion: Stress, Intensity: High") to the generative AI model. This feedback process allows the generative AI model to reflect the user's emotional state in the generation process of the assistance method.

[1337] Step 6:

[1338] Running generative AI models

[1339] The server's generative AI model generates optimal support methods and practical examples based on the preprocessed input data and the fed-back emotional data. Using machine learning algorithms, the generative AI model suggests, for example, "communication practice using picture cards." It also provides advice such as "It is recommended to practice in an environment where the user can relax."

[1340] Step 7:

[1341] Processing and sending results

[1342] The server organizes the generated support methods and practice examples and converts them back into structured data in JSON format. The converted data package (e.g., "{\"Support method\": \"Communication practice using picture cards\", \"Recommended situation\": \"Perform in a relaxed environment\"}") is sent to the device using the HTTPS protocol.

[1343] Step 8:

[1344] Displaying the results

[1345] The device retrieves the data received from the server and displays it in a format that is easy for the user to understand. For example, information such as "Communication support method: Use of picture cards. Practical example: Successful example at XX school" is displayed on the screen. Based on this information, the user can quickly implement specific support methods.

[1346] (Application example 2)

[1347] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1348] Autonomous vehicles lack the ability to properly understand the emotional state of pedestrians and provide appropriate assistance based on that understanding. As a result, pedestrian safety and a sense of security cannot be fully ensured. Furthermore, conventional autonomous driving systems have difficulty adapting flexibly to the surrounding situation, and communication with pedestrians in particular is lacking.

[1349] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1350] In this invention, the server includes means for inputting information about the difficulties faced by an individual with a disability, means for transmitting the input information to the server, means for the server to receive the input information and search for an optimal assistance method using a generative AI model and an emotion engine, means for the server to transmit the assistance method and practical example generated using the generative AI model and emotion engine to a terminal, means for the terminal to display the transmitted assistance method and practical example, means for the system to collect video and audio data from a camera and microphone mounted on a mobile vehicle and analyze the emotional state from the data, and means for providing appropriate assistance methods to nearby users based on the emotional data analyzed by the generative AI model. This enables flexible and appropriate assistance to ensure safety and security by appropriately understanding the emotional state of pedestrians.

[1351] "Information on the challenges faced by individuals with disabilities" is data that documents the specific problems and difficulties faced by individuals with disabilities.

[1352] The "server" is a high-performance computer that processes data received from user terminals via the network and generates and distributes the necessary support methods.

[1353] A "generative AI model" is a machine learning algorithm that suggests optimal support methods based on information input by the user.

[1354] The "emotion engine" is a program that analyzes the user's emotions from video and audio data and provides appropriate feedback to the generative AI model.

[1355] A "terminal" is an electronic device that allows a user to input information and receive and display the generated support methods and practical examples.

[1356] "Mobile object" refers to a moving mechanical device such as an autonomous vehicle, equipped with sensors to sense its surroundings.

[1357] A "camera" is a photographic device used to collect video data and record the facial expressions and movements of pedestrians.

[1358] A "microphone" is a device for collecting audio data and is used to record surrounding sounds and speech.

[1359] "Video data" refers to visual information collected by a camera in the form of images or videos that are subject to analysis.

[1360] "Audio Data" means audio information collected by a microphone in the form of sound or speech that is subject to analysis.

[1361] The "means for analyzing emotional state" refers to a program or algorithm for determining the user's emotions based on video and audio data.

[1362] The "means for providing an appropriate support method" is a program or device for providing the user with the most appropriate response based on the analyzed emotional state.

[1363] A "structured data format" is a format for arranging data into a unified and easy-to-use format.

[1364] The specific system configuration and operation procedure for an embodiment of the present invention are described below. The system is broadly composed of a user terminal, a server, a generative AI model, an emotion engine, and a mobile object (an autonomously driven vehicle).

[1365] User terminal configuration and functions

[1366] A user terminal is an electronic device for inputting information about the difficulties faced by individuals with disabilities. The terminal provides an information input interface that allows users to input the type of disability and specific difficulties. Specifically, the terminal has an interface for typing information such as autism spectrum disorder and difficulties with social interaction.

[1367] Server configuration and functions

[1368] The server receives the information sent from the user device, analyzes the data, and executes the generative AI model. The server uses the following software in particular:

[1369] TensorFlow: Running Generative AI Models

[1370] OpenCV: Video data preprocessing and analysis

[1371] EmotionAPI: Emotion Engine

[1372] The server analyzes the data sent from the user's device and generates optimal support methods and related practical examples using a generative AI model. Furthermore, it appropriately adjusts the generated support methods based on the emotional data analyzed by the emotion engine.

[1373] Linking generative AI models and emotion engines

[1374] The generative AI model is a machine learning algorithm that generates assistance methods based on information and emotional data input by the user. During this process, the emotion engine analyzes the user's emotional state and provides feedback to the generative AI model to provide more appropriate assistance methods.

[1375] Mobile device configuration and functions

[1376] This system uses cameras and microphones mounted on moving objects (autonomous vehicles) to collect video and audio data in real time. The emotion engine analyzes this data and determines the emotional state of pedestrians. The generative AI model provides appropriate assistance methods to nearby users based on the analyzed emotional data.

[1377] As a concrete example, consider a situation where a pedestrian stands at a crosswalk while an autonomous vehicle is traveling down the road. At this time, the system captures the pedestrian's facial expressions with a camera, and the emotion engine analyzes whether the pedestrian is feeling "tension" or "stress." Based on this emotional data, the generative AI model suggests "stopping the vehicle" and notifies the driver via a display and audio alert saying, "Please be careful, pedestrians have the right of way." This allows the pedestrian to cross the road safely.

[1378] Prompt Sentence Examples

[1379] Based on the input data from the camera and microphone installed in the autonomous vehicle, determine whether the pedestrian is feeling stressed and generate the optimal response method based on the result. For example, if the pedestrian is feeling stressed, consider a response such as "the vehicle stopping and encouraging the pedestrian to cross first."

[1380] As a result, the present invention enables autonomous vehicles to flexibly adapt to their surroundings, ensuring safety and a sense of security for pedestrians.

[1381] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1382] Step 1: Collect data

[1383] Cameras and microphones installed in autonomous vehicles collect video and audio data of the surrounding area in real time. The cameras capture pedestrians' faces and gestures, and the microphones record environmental sounds and pedestrians' voices. The collected video and audio data become input to the system.

[1384] Step 2: Preprocessing the data

[1385] The server uses OpenCV to identify faces and facial expressions from the collected video data. Specifically, the video is converted to grayscale and a face detection algorithm is applied. Meanwhile, the audio data is denoised and normalised to extract acoustic features. This pre-processing results in clean data for analysis.

[1386] Step 3: Sentiment Analysis

[1387] The server uses the Emotion API to analyze the pedestrian's emotional state from preprocessed video and audio data. The preprocessed video and audio are used as input, and the pedestrian's emotional state (e.g., tension, stress, relief) is returned as output. Specifically, emotions are determined from subtle changes in facial expressions and voice tone.

[1388] Step 4: Generating assistance methods using generative AI models

[1389] The server inputs the analyzed emotion data into a generative AI model using TensorFlow. This model generates the optimal assistance method by taking into account the pedestrian's emotion. The input is the pedestrian's emotion data, and the output is the appropriate countermeasure (e.g., stopping the vehicle, audio notification).

[1390] Step 5: Communicate how to help

[1391] The server sends the generated assistance method to the display and voice system of the autonomous vehicle. The display will show a message saying "Please be careful, pedestrians have the right of way," and the voice system will make a similar announcement. This allows the pedestrian to receive immediate and appropriate assistance.

[1392] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1393] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1394] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1395] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1396] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1397] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1398] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1399] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1400] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1401] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1402] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1403] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1404] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1405] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1406] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1407] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1408] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1409] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1410] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1411] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1412] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1413] The following is further disclosed regarding the above embodiment.

[1414] (Claim 1)

[1415] a means for inputting information about the challenges faced by individuals with disabilities; and

[1416] means for transmitting the input information to a server;

[1417] A means for the server to receive the input information and search for an optimal support method using a generative AI model;

[1418] A means for transmitting the support method and practical examples generated by the server using the generative AI model to a terminal;

[1419] a means for displaying the support method and practical examples transmitted by the terminal;

[1420] A system including:

[1421] (Claim 2)

[1422] 10. The system of claim 1, further comprising means for preprocessing the input information.

[1423] (Claim 3)

[1424] The system of claim 1, further comprising means for transmitting the generated support method and practical examples in a structured data format.

[1425] "Example 1"

[1426] (Claim 1)

[1427] A means to input information about children's disabilities and difficulties,

[1428] means for converting the input information into structured data and transmitting the structured data to a server using a secure communication protocol;

[1429] a means for the server to receive the input information and perform preprocessing such as text normalization and tokenization;

[1430] A means for the server to generate optimal support methods and practical examples from preprocessed information using a generation AI model;

[1431] a means for transmitting the generated support method and practical examples to a terminal in a structured data format by the server;

[1432] a means for displaying the support method and practical examples transmitted by the terminal in a format that is easy for the user to understand;

[1433] A system including:

[1434] (Claim 2)

[1435] 2. The system of claim 1, wherein the terminal further comprises means for adding a prompt sentence from the user as part of the information to be input.

[1436] (Claim 3)

[1437] 10. The system of claim 1, further comprising means for preprocessing including text normalization, tokenization, and checking data consistency and integrity.

[1438] "Application Example 1"

[1439] (Claim 1)

[1440] a means for inputting information about the challenges faced by individuals with disabilities; and

[1441] means for transmitting the input information to a server;

[1442] A means for the server to receive the input information and search for an optimal support method using a generative AI model;

[1443] A means for transmitting the support method and practical examples generated by the server using the generative AI model to a terminal;

[1444] a means for displaying the support method and practical examples transmitted by the terminal;

[1445] a means for the terminal to input factory work efficiency improvements and problem-solving methods and generate optimal work procedures using a generative AI model;

[1446] A means for transmitting examples related to the work procedures generated by the server using the generative AI model to a terminal;

[1447] a means for displaying the transmitted work procedure and related cases on the terminal;

[1448] A system including:

[1449] (Claim 2)

[1450] 10. The system of claim 1, further comprising means for preprocessing the input information.

[1451] (Claim 3)

[1452] The system of claim 1, further comprising means for transmitting the generated support method and practical examples in a structured data format.

[1453] "Example 2: Combining Emotion Engines"

[1454] (Claim 1)

[1455] a means for inputting information about the challenges faced by individuals with disabilities; and

[1456] means for transmitting the input information to a server;

[1457] A means for the server to receive the input information and generate an optimal assistance method using a generative AI model;

[1458] A means for the server to acquire emotion data using an emotion engine that analyzes the emotion data of the user and feeds the emotion data back to the generative AI model;

[1459] A means for transmitting the support method and practical examples generated by the server using the generative AI model to a terminal;

[1460] a means for displaying the support method and practical examples transmitted by the terminal;

[1461] A system including:

[1462] (Claim 2)

[1463] 10. The system of claim 1, further comprising: means for preprocessing the input information; and means for converting the preprocessed information into a structured data format.

[1464] (Claim 3)

[1465] The system of claim 1, further comprising means for transmitting the generated support method and practice examples using a secure protocol.

[1466] "Application example 2 when combining emotion engines"

[1467] (Claim 1)

[1468] a means for inputting information about the challenges faced by individuals with disabilities; and

[1469] means for transmitting the input information to a server;

[1470] A means for the server to receive the input information and search for an optimal support method using a generative AI model and an emotion engine;

[1471] A means for transmitting the support method and practical examples generated by the server using the generative AI model and emotion engine to a terminal;

[1472] a means for displaying the support method and practical examples transmitted by the terminal;

[1473] The system includes means for collecting video and audio data from a camera and a microphone mounted on the vehicle and analyzing the emotional state from the data;

[1474] A means for providing an appropriate support method to nearby users based on the emotion data analyzed by the generative AI model;

[1475] A system including:

[1476] (Claim 2)

[1477] 10. The system of claim 1, further comprising means for preprocessing the input information.

[1478] (Claim 3)

[1479] The system of claim 1, further comprising means for transmitting the generated support method and practical examples in a structured data format. [Explanation of symbols]

[1480] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for inputting information about the challenges faced by individuals with disabilities; and means for transmitting the input information to a server; A means for the server to receive the input information and search for an optimal support method using a generative AI model; A means for transmitting the support method and practical examples generated by the server using the generative AI model to a terminal; a means for displaying the support method and practical examples transmitted by the terminal; A system including:

2. 2. The system of claim 1, further comprising means for preprocessing the input information.

3. The system according to claim 1 , further comprising means for transmitting the generated support method and practical examples in a structured data format.

Images