System

Abstract

An object of a system according to an embodiment is to update devices and expand a talk language using a generated AI and to facilitate communication between a speaker and a listener.SOLUTION: A system includes a device updating unit, a talk language expansion unit, and a communication facilitation unit. The updating unit updates the devices using the generated AI. The talk language expansion unit expands the talk language using the generated AI. The communication smoothening unit smoothens communication between the speaker and the listener using the generated AI.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has not adequately updated the devices and speech languages ​​to facilitate communication between deaf and hearing people, and there is room for improvement.

[0005] The system of this embodiment aims to facilitate communication between deaf and hearing people by using generative AI to update devices and expand talk languages. [Means for solving the problem]

[0006] The system according to the embodiment includes a device updating unit, a talk language enhancement unit, and a communication facilitation unit. The device updating unit updates the device using a generation AI. The talk language enhancement unit enhances the talk language using the generation AI. The communication facilitation unit facilitates communication between deaf and hearing people using the generation AI. [Effects of the Invention]

[0007] The system of the embodiment uses generative AI to update devices and expand talk languages, facilitating communication between deaf and hearing people. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The SureTalk system according to the embodiment of the present invention is a system that uses generation AI and various generation AIs to update devices and expand talk languages, facilitating communication between deaf and hearing people. As a result, the SureTalk system will make communication between deaf and hearing people borderless, and will take advantage of the opportunity of DeafLympuk to expand its global market share.

[0029] The SureTalk system according to the embodiment includes a device modernization unit, a talk language enhancement unit, and a communication facilitation unit. The device modernization unit uses a generation AI to upgrade the device. For example, the generation AI optimizes the placement of the device's sensors and microphones. The generation AI can also optimize the device to extend its battery life. The generation AI can also design the device to be lightweight. The talk language enhancement unit uses the generation AI to enhance the talk language. For example, the generation AI analyzes prompts input by a user and generates an operating program that translates them into an appropriate language or sign language. The generation AI can also generate a program for multilingual support. The generation AI can also generate a program for recognizing and translating sign language and gestures. The communication facilitation unit uses the generation AI to facilitate communication between deaf and hearing people. For example, the generation AI converts what a deaf person says in sign language into speech in real time and transmits it to a hearing person. The generation AI can also convert what a hearing person says into sign language in real time and transmit it to a deaf person. The generation AI can also generate a program for facilitating communication between deaf and hearing people. As a result, the SureTalk system according to the embodiment realizes the modernization of devices, the expansion of talk languages, and smoother communication between deaf and hearing people.

[0030] The device modernization unit can use generative AI to optimize the placement of device sensors and microphones. For example, the device modernization unit uses generative AI to analyze user usage patterns and optimize the device's battery consumption. For example, it automatically configures settings to reduce battery consumption during times of high usage. The generative AI also learns the user's usage history and automatically customizes device settings. For example, it prioritizes displaying functions that the user uses frequently. The generative AI also analyzes user behavior patterns and simplifies device operation. For example, automating certain operations reduces the burden on the user. This optimizes the placement of the device's sensors and microphones and improves performance.

[0031] The talk language expansion unit can use the generation AI to analyze prompts input by the user and generate an operating program that translates them into an appropriate language or sign language. In the talk language expansion unit, for example, the generation AI analyzes the user's pronunciation and accent and provides an individually optimized translation. For example, it learns the user's pronunciation characteristics and performs an appropriate translation. The generation AI also collects the user's voice data and analyzes the characteristics of pronunciation and accent. For example, it provides the optimal translation for a user with a specific accent. The generation AI also analyzes the user's pronunciation and accent in real time and reflects this in the translation results. For example, it performs a translation that matches the user's pronunciation. In this way, an operating program is generated that translates into an appropriate language or sign language based on the user's prompt.

[0032] The communication facilitation unit uses generation AI to convert what deaf people say in sign language into speech in real time and convey it to hearing people. The communication facilitation unit, for example, uses an emotion estimation function to detect the user's stress level and simplify device operation. For example, it uses the emotion estimation function to detect the stress level from the user's facial expressions and voice. For example, it simplifies device operation when stress is high. It also uses the emotion estimation function to provide an interface according to the user's stress level. For example, it displays a simple operation screen when stress is high. It also uses the emotion estimation function to monitor the user's stress level in real time and provide appropriate feedback. For example, it displays advice on how to relax. This makes it possible to convert what deaf people say in sign language into speech in real time and convey it to hearing people.

[0033] The communication facilitation unit uses generation AI to convert what a hearing person says into sign language in real time and communicate it to a deaf person. For example, the communication facilitation unit uses generation AI to analyze a user's communication history and suggest appropriate responses based on the content of past conversations. For example, the generation AI analyzes a user's communication history and suggests appropriate responses based on the content of past conversations. For example, it refers to the content of previous conversations and provides relevant responses. The generation AI also learns the user's communication patterns and automatically generates appropriate responses. For example, it learns the user's speaking style and expressions and provides natural responses. The generation AI also suggests responses tailored to the user's preferences and interests based on past conversation data. For example, it provides responses related to topics that interest the user. This allows what a hearing person says to be converted into sign language in real time and communicated to a deaf person.

[0034] The device modernization unit can use generative AI to analyze user usage patterns and add automatic device adjustment functions. For example, the device modernization unit uses generative AI to analyze user usage patterns and optimize device battery consumption. For example, it automatically configures settings to reduce battery consumption during times of high usage. The generative AI also learns the user's usage history and automatically customizes device settings. For example, it prioritizes displaying functions that the user uses frequently. The generative AI also analyzes user behavior patterns and simplifies device operation. For example, automating certain operations reduces the burden on the user. This makes it possible to automatically adjust the device based on the user's usage patterns.

[0035] The device modernization unit can add a function that uses generative AI to predict device failures and suggest maintenance in advance. For example, the generative AI analyzes data obtained from the device's sensors to detect signs of failure. For example, it detects abnormalities in temperature or vibration and suggests maintenance to the user. The generative AI also learns from past failure data and builds a failure prediction model. For example, if a specific pattern is detected, it will display a warning to the user. The generative AI also monitors the device status in real time and immediately notifies the user when an abnormality occurs. For example, it detects battery degradation and suggests replacement. This makes it possible to predict device failures and perform pre-maintenance.

[0036] The device modernization unit can add an environmental sensor to the device and automatically adjust the sensitivity of voice recognition according to the ambient noise level. For example, the device modernization unit can equip the device with a noise sensor and measure the ambient noise level in real time. For example, it can increase the sensitivity of voice recognition when the noise level is loud. It can also use the environmental sensor to analyze the ambient sound environment and automatically set the optimal voice recognition settings. For example, it can set the sensitivity low in a quiet environment. The device can also adjust the filtering of voice recognition according to the ambient noise level. For example, it can strengthen noise canceling when there is a lot of noise. This allows the sensitivity of voice recognition to be automatically adjusted according to the ambient noise level.

[0037] The device modernization unit can make the device wearable and position it in the optimal position according to the user's movements. The device modernization unit, for example, makes the device wearable and automatically adjusts it to the optimal position according to the user's movements. For example, it is worn on the arm or neck and changes its position according to the movement. The wearable device also detects the user's movements in real time and positions it in the optimal position. For example, it moves the device to a stable position during exercise. The wearable device also analyzes the user's posture and movements and suggests the optimal position. For example, it positions it differently when sitting and when standing. In this way, the device is made wearable and positions it in the optimal position according to the user's movements.

[0038] The talk language enhancement unit can use the generation AI to enhance the talk language so that it can accommodate regional dialects and slang. For example, the generation AI in the talk language enhancement unit learns regional dialects and slang and reflects them in the talk language. For example, it recognizes phrases used in a specific region and reflects them in the translation. The generation AI also analyzes user input and automatically detects dialects and slang. For example, it recognizes specific words and phrases as dialects and translates them appropriately. The generation AI also collects regional language data and enhances the talk language database. For example, it learns regional linguistic characteristics and improves translation accuracy. This enhances the talk language to accommodate regional dialects and slang.

[0039] The talk language enhancement unit can use the generation AI to learn the user's pronunciation and accent and provide individually optimized translations. For example, the generation AI analyzes the user's pronunciation and accent and provides individually optimized translations. For example, it learns the user's pronunciation characteristics and provides appropriate translations. The generation AI also collects the user's voice data and analyzes the characteristics of pronunciation and accent. For example, it provides the optimal translation for a user with a specific accent. The generation AI also analyzes the user's pronunciation and accent in real time and reflects this in the translation results. For example, it provides a translation that matches the user's pronunciation. This provides a translation that is optimized based on the user's pronunciation and accent.

[0040] In addition to enhancing the talk language, the talk language enhancement unit also recognizes cultural background and gestures, enabling more natural communication. In the talk language enhancement unit, for example, the generation AI recognizes cultural background and gestures and reflects them in the talk language. For example, it understands gestures in a particular culture and translates them appropriately. The generation AI also analyzes the user's gestures and reflects them in the talk language. For example, it recognizes hand movements and facial expressions to enable more natural communication. The generation AI also learns cultural background and enhances the talk language database. For example, it understands linguistic characteristics in different cultures and improves translation accuracy. This recognizes cultural background and gestures, enabling more natural communication.

[0041] The talk language enhancement unit can use the generation AI to add customization functions according to the speed and style of sign language movements. For example, the generation AI analyzes the speed and style of sign language movements and provides customized translations. For example, it performs translations that match the speed of the user's sign language. The generation AI also learns the sign language style and provides individually optimized translations. For example, it performs appropriate translations for users with a specific sign language style. The generation AI also analyzes the speed and style of sign language movements in real time and reflects this in the translation results. For example, it performs translations that match the user's sign language. This adds customization functions according to the speed and style of sign language movements.

[0042] The communication facilitation unit can add a function that uses the generation AI to translate non-verbal communication between deaf and hearing people. For example, the generation AI analyzes facial expressions and gestures to translate non-verbal communication. For example, it recognizes smiles and hand movements and provides an appropriate translation. The generation AI also analyzes the user's non-verbal communication in real time and reflects this in the translation results. For example, it provides translations that match the user's facial expressions. The generation AI also builds a database of non-verbal communication to improve translation accuracy. For example, it learns different gestures and facial expressions and translates them appropriately. This allows non-verbal communication between deaf and hearing people to be translated.

[0043] The communication facilitation unit can use the generation AI to learn the user's communication history and suggest appropriate responses based on the content of past conversations. For example, the communication facilitation unit uses the generation AI to analyze the user's communication history and suggest appropriate responses based on the content of past conversations. For example, it refers to the content of previous conversations and provides relevant responses. The generation AI also learns the user's communication patterns and automatically generates appropriate responses. For example, it learns the user's speaking style and expressions and provides natural responses. The generation AI also suggests responses tailored to the user's preferences and interests based on past conversation data. For example, it provides responses related to topics that interest the user. In this way, appropriate responses are suggested based on the user's communication history.

[0044] The communication facilitation unit uses generative AI to support communication between deaf and hearing people, not only translating in real time but also understanding the context of the conversation and providing appropriate advice. For example, the generative AI analyzes the context of the conversation and provides appropriate advice. For example, it understands the flow of the conversation and suggests what to say next. The generative AI also understands the context of the conversation while translating in real time and provides appropriate feedback. For example, it provides information related to the topic of the conversation. The generative AI also learns the context of the conversation and provides appropriate advice to the user. For example, it suggests what to say next as the conversation progresses. In this way, understanding the context of the conversation and providing appropriate advice facilitates communication between deaf and hearing people.

[0045] The communication facilitation unit can use the generative AI to add an interactive whiteboard function that can be used jointly by deaf and hearing people. For example, the generative AI provides an interactive whiteboard function that can be used jointly by deaf and hearing people. For example, handwritten notes and drawings can be shared in real time. The generative AI can also translate the content on the whiteboard in real time to enable smooth communication between deaf and hearing people. For example, it can convert handwritten characters into speech. The generative AI can also use the interactive whiteboard function to provide a tool that allows deaf and hearing people to work together on a project. For example, they can jointly brainstorm ideas and share them in real time. This adds an interactive whiteboard function that can be used jointly by deaf and hearing people.

[0046] The device modernization unit uses generation AI to monitor usage status on Defrympook in real time and make immediate improvements. For example, the device modernization unit uses generation AI to monitor usage status on Defrympook in real time and make immediate improvements. For example, it adjusts settings based on user feedback. The generation AI also analyzes usage data on Defrympook and immediately suggests improvements. For example, it prioritizes improvements to frequently used functions. The generation AI also monitors usage status on Defrympook in real time and responds immediately if an abnormality occurs. For example, it detects and corrects system malfunctions. This allows usage status on Defrympook to be monitored in real time and make immediate improvements.

[0047] The device modernization unit can use generation AI to analyze user feedback on Defrimpuk and automatically suggest improvements for the next event. For example, the device modernization unit uses generation AI to analyze user feedback on Defrimpuk and automatically suggest improvements for the next event. For example, new functions are added based on user opinions. The generation AI also collects user feedback data and suggests improvements based on the analysis results. For example, it suggests improvements to resolve user dissatisfaction. The generation AI also analyzes feedback on Defrimpuk in real time and immediately suggests improvements. For example, it adds functions in response to user requests. In this way, user feedback on Defrimpuk is analyzed and improvements for the next event are automatically suggested.

[0048] Based on the success story at DeFrimpuk, the Device Modernization Department can introduce SureTalk at other international events and exhibitions to promote its use. Based on the success story at DeFrimpuk, the Device Modernization Department can introduce SureTalk at other international events and exhibitions to promote its use. For example, they can introduce the success story in a presentation to promote its use. They can also analyze the usage data at DeFrimpuk and develop a strategy for introducing SureTalk at other events. For example, they can analyze the success factors and propose how to apply it at other events. They can also use the success story at DeFrimpuk to promote SureTalk. For example, they can launch a marketing campaign based on the success story. In this way, based on the success story at DeFrimpuk, the Device Modernization Department can introduce SureTalk at other international events and exhibitions to promote its use.

[0049] The device modernization unit can use generative AI to analyze usage data from Defrinpook and develop versions adapted to different markets and cultural spheres. For example, the device modernization unit uses generative AI to analyze usage data from Defrinpook and develop versions adapted to different markets and cultural spheres. For example, functions tailored to specific cultures can be added. The generative AI also collects user data from Defrinpook and develops market-specific versions based on the analysis results. For example, customization can be performed according to regional needs. The generative AI also analyzes usage status on Defrinpook and proposes versions adapted to different markets and cultural spheres. For example, providing designs and functions tailored to specific markets. This makes it possible to analyze usage data from Defrinpook and develop versions adapted to different markets and cultural spheres.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The SureTalk system can also be equipped with a health management unit that monitors the user's health status. For example, the health management unit can measure the user's heart rate and blood pressure in real time and issue an alert if an abnormality is detected. The health management unit can also analyze the user's sleep patterns and provide advice to promote quality sleep. Furthermore, the health management unit can record the user's exercise volume and propose an exercise plan to maintain health. In this way, the SureTalk system can support the user's health management and provide a more fulfilling life.

[0052] The device updater can also utilize the user's location information to provide an optimal communication environment. For example, it can analyze the communication conditions at the user's location based on the location information and automatically configure the optimal network settings. It can also utilize the location information to make adjustments to maintain stable communication even while the user is on the move. It can also use the location information to provide customized services tailored to the locations the user visits. This improves the device's communication performance and increases user convenience.

[0053] The Talk Language Enhancement Unit can also provide an individually optimized learning plan based on the user's learning history. For example, it can analyze what the user has learned in the past and suggest what they should learn next. It can also monitor the user's learning pace and level of understanding in real time and provide appropriate feedback. It can also customize learning content based on the user's interests. This improves the user's learning efficiency and makes learning more effective.

[0054] The SureTalk system can also include a content provider that provides customized content based on the user's hobbies and interests. For example, it can automatically collect and provide news and articles related to topics that interest the user. It can also suggest events and activities related to the user's hobbies. It can also provide customized entertainment content based on the user's interests. This allows the user to enjoy a more fulfilling life by receiving content tailored to their interests.

[0055] The device modernization module can further analyze user usage and optimize energy efficiency. For example, it can automatically turn off infrequently used features to reduce battery consumption. It can also suggest energy-efficient settings based on the user's usage patterns. It can also optimize device thermal management to reduce energy consumption. This improves the device's energy efficiency and extends battery life.

[0056] The SureTalk system can also be equipped with a reminder section that analyzes the user's behavioral patterns and provides appropriate reminders. For example, if a user often performs a specific action at a specific time, a reminder to encourage that action can be displayed. It can also remind the user of important appointments based on the user's schedule. It can also provide reminders for health management and task management based on the user's behavioral patterns. This provides reminders that are tailored to the user's behavioral patterns, leading to a more efficient lifestyle.

[0057] The processing flow of the first embodiment will be briefly explained below.

[0058] Step 1: The device modernization unit uses generative AI to modernize the device. For example, generative AI can optimize the placement of sensors and microphones on the device. Generative AI can also optimize the device to extend its battery life. Generative AI can also design the device to be lightweight. Step 2: The talk language expansion unit uses the generation AI to expand the talk language. For example, the generation AI analyzes prompts entered by the user and generates an operating program that translates them into an appropriate language or sign language. The generation AI can also generate programs that enable multilingual support. Furthermore, the generation AI can generate programs that recognize and translate sign language and gestures. Step 3: The communication facilitation unit uses the generation AI to facilitate communication between deaf and hearing people. For example, the generation AI can convert what a deaf person says in sign language into audio in real time and convey it to a hearing person. The generation AI can also convert what a hearing person says into sign language in real time and convey it to a deaf person. Furthermore, the generation AI can generate programs to facilitate smooth communication between deaf and hearing people.

[0059] (Example 2) The SureTalk system according to the embodiment of the present invention is a system that uses generation AI and various generation AIs to update devices and expand talk languages, facilitating communication between deaf and hearing people. As a result, the SureTalk system will make communication between deaf and hearing people borderless, and will take advantage of the opportunity of DeafLympuk to expand its global market share.

[0060] The SureTalk system according to the embodiment includes a device modernization unit, a talk language enhancement unit, and a communication facilitation unit. The device modernization unit uses a generation AI to upgrade the device. For example, the generation AI optimizes the placement of the device's sensors and microphones. The generation AI can also optimize the device to extend its battery life. The generation AI can also design the device to be lightweight. The talk language enhancement unit uses the generation AI to enhance the talk language. For example, the generation AI analyzes prompts input by a user and generates an operating program that translates them into an appropriate language or sign language. The generation AI can also generate a program for multilingual support. The generation AI can also generate a program for recognizing and translating sign language and gestures. The communication facilitation unit uses the generation AI to facilitate communication between deaf and hearing people. For example, the generation AI converts what a deaf person says in sign language into speech in real time and transmits it to a hearing person. The generation AI can also convert what a hearing person says into sign language in real time and transmit it to a deaf person. The generation AI can also generate a program for facilitating communication between deaf and hearing people. As a result, the SureTalk system according to the embodiment realizes the modernization of devices, the expansion of talk languages, and smoother communication between deaf and hearing people.

[0061] The device modernization unit can use generative AI to optimize the placement of device sensors and microphones. For example, the device modernization unit uses generative AI to analyze user usage patterns and optimize the device's battery consumption. For example, it automatically configures settings to reduce battery consumption during times of high usage. The generative AI also learns the user's usage history and automatically customizes device settings. For example, it prioritizes displaying functions that the user uses frequently. The generative AI also analyzes user behavior patterns and simplifies device operation. For example, automating certain operations reduces the burden on the user. This optimizes the placement of the device's sensors and microphones and improves performance.

[0062] The talk language expansion unit can use the generation AI to analyze prompts input by the user and generate an operating program that translates them into an appropriate language or sign language. In the talk language expansion unit, for example, the generation AI analyzes the user's pronunciation and accent and provides an individually optimized translation. For example, it learns the user's pronunciation characteristics and performs an appropriate translation. The generation AI also collects the user's voice data and analyzes the characteristics of pronunciation and accent. For example, it provides the optimal translation for a user with a specific accent. The generation AI also analyzes the user's pronunciation and accent in real time and reflects this in the translation results. For example, it performs a translation that matches the user's pronunciation. In this way, an operating program is generated that translates into an appropriate language or sign language based on the user's prompt.

[0063] The communication facilitation unit uses generation AI to convert what deaf people say in sign language into speech in real time and convey it to hearing people. The communication facilitation unit, for example, uses an emotion estimation function to detect the user's stress level and simplify device operation. For example, it uses the emotion estimation function to detect the stress level from the user's facial expressions and voice. For example, it simplifies device operation when stress is high. It also uses the emotion estimation function to provide an interface according to the user's stress level. For example, it displays a simple operation screen when stress is high. It also uses the emotion estimation function to monitor the user's stress level in real time and provide appropriate feedback. For example, it displays advice on how to relax. This makes it possible to convert what deaf people say in sign language into speech in real time and convey it to hearing people.

[0064] The communication facilitation unit uses generation AI to convert what a hearing person says into sign language in real time and communicate it to a deaf person. For example, the communication facilitation unit uses generation AI to analyze a user's communication history and suggest appropriate responses based on the content of past conversations. For example, the generation AI analyzes a user's communication history and suggests appropriate responses based on the content of past conversations. For example, it refers to the content of previous conversations and provides relevant responses. The generation AI also learns the user's communication patterns and automatically generates appropriate responses. For example, it learns the user's speaking style and expressions and provides natural responses. The generation AI also suggests responses tailored to the user's preferences and interests based on past conversation data. For example, it provides responses related to topics that interest the user. This allows what a hearing person says to be converted into sign language in real time and communicated to a deaf person.

[0065] The device modernization unit can use generative AI to analyze user usage patterns and add automatic device adjustment functions. For example, the device modernization unit uses generative AI to analyze user usage patterns and optimize device battery consumption. For example, it automatically configures settings to reduce battery consumption during times of high usage. The generative AI also learns the user's usage history and automatically customizes device settings. For example, it prioritizes displaying functions that the user uses frequently. The generative AI also analyzes user behavior patterns and simplifies device operation. For example, automating certain operations reduces the burden on the user. This makes it possible to automatically adjust the device based on the user's usage patterns.

[0066] The device modernization unit can add a function that uses generative AI to predict device failures and suggest maintenance in advance. For example, the generative AI analyzes data obtained from the device's sensors to detect signs of failure. For example, it detects abnormalities in temperature or vibration and suggests maintenance to the user. The generative AI also learns from past failure data and builds a failure prediction model. For example, if a specific pattern is detected, it will display a warning to the user. The generative AI also monitors the device status in real time and immediately notifies the user when an abnormality occurs. For example, it detects battery degradation and suggests replacement. This makes it possible to predict device failures and perform pre-maintenance.

[0067] The device modernization unit can use the emotion estimation function to detect the user's stress level and simplify device operation. The device modernization unit, for example, uses the emotion estimation function to detect the stress level from the user's facial expression or voice. For example, if stress is high, the device operation is simplified. The emotion estimation function is also used to provide an interface according to the user's stress level. For example, if stress is high, a simple operation screen is displayed. The emotion estimation function is also used to monitor the user's stress level in real time and provide appropriate feedback. For example, advice on how to relax is displayed. This simplifies device operation according to the user's stress level.

[0068] The device modernization unit can add an environmental sensor to the device and automatically adjust the sensitivity of voice recognition according to the ambient noise level. For example, the device modernization unit can equip the device with a noise sensor and measure the ambient noise level in real time. For example, it can increase the sensitivity of voice recognition when the noise level is loud. It can also use the environmental sensor to analyze the ambient sound environment and automatically set the optimal voice recognition settings. For example, it can set the sensitivity low in a quiet environment. The device can also adjust the filtering of voice recognition according to the ambient noise level. For example, it can strengthen noise canceling when there is a lot of noise. This allows the sensitivity of voice recognition to be automatically adjusted according to the ambient noise level.

[0069] The device modernization unit can make the device wearable and position it in the optimal position according to the user's movements. The device modernization unit, for example, makes the device wearable and automatically adjusts it to the optimal position according to the user's movements. For example, it is worn on the arm or neck and changes its position according to the movement. The wearable device also detects the user's movements in real time and positions it in the optimal position. For example, it moves the device to a stable position during exercise. The wearable device also analyzes the user's posture and movements and suggests the optimal position. For example, it positions it differently when sitting and when standing. In this way, the device is made wearable and positions it in the optimal position according to the user's movements.

[0070] The talk language enhancement unit can use the generation AI to enhance the talk language so that it can accommodate regional dialects and slang. For example, the generation AI in the talk language enhancement unit learns regional dialects and slang and reflects them in the talk language. For example, it recognizes phrases used in a specific region and reflects them in the translation. The generation AI also analyzes user input and automatically detects dialects and slang. For example, it recognizes specific words and phrases as dialects and translates them appropriately. The generation AI also collects regional language data and enhances the talk language database. For example, it learns regional linguistic characteristics and improves translation accuracy. This enhances the talk language to accommodate regional dialects and slang.

[0071] The talk language enhancement unit can use the generation AI to learn the user's pronunciation and accent and provide individually optimized translations. For example, the generation AI analyzes the user's pronunciation and accent and provides individually optimized translations. For example, it learns the user's pronunciation characteristics and provides appropriate translations. The generation AI also collects the user's voice data and analyzes the characteristics of pronunciation and accent. For example, it provides the optimal translation for a user with a specific accent. The generation AI also analyzes the user's pronunciation and accent in real time and reflects this in the translation results. For example, it provides a translation that matches the user's pronunciation. This provides a translation that is optimized based on the user's pronunciation and accent.

[0072] The talk language enhancement unit can use the emotion estimation function to perform translation that reflects the tone and nuance according to the user's emotion. The talk language enhancement unit, for example, uses the emotion estimation function to perform translation that reflects the tone and nuance according to the user's emotion. For example, if the user is angry, the translation is performed in an appropriate tone. The emotion estimation function is also used to analyze the user's emotion and reflect this in the translation result. For example, if the user is happy, a translation with a positive nuance is performed. The emotion estimation function is also used to provide a translation that corresponds to the user's emotion. For example, if the user is sad, the translation is performed in a gentle tone. In this way, a translation that reflects the tone and nuance according to the user's emotion is performed.

[0073] In addition to enhancing the talk language, the talk language enhancement unit also recognizes cultural background and gestures, enabling more natural communication. In the talk language enhancement unit, for example, the generation AI recognizes cultural background and gestures and reflects them in the talk language. For example, it understands gestures in a particular culture and translates them appropriately. The generation AI also analyzes the user's gestures and reflects them in the talk language. For example, it recognizes hand movements and facial expressions to enable more natural communication. The generation AI also learns cultural background and enhances the talk language database. For example, it understands linguistic characteristics in different cultures and improves translation accuracy. This recognizes cultural background and gestures, enabling more natural communication.

[0074] The talk language enhancement unit can use the generation AI to add customization functions according to the speed and style of sign language movements. For example, the generation AI analyzes the speed and style of sign language movements and provides customized translations. For example, it performs translations that match the speed of the user's sign language. The generation AI also learns the sign language style and provides individually optimized translations. For example, it performs appropriate translations for users with a specific sign language style. The generation AI also analyzes the speed and style of sign language movements in real time and reflects this in the translation results. For example, it performs translations that match the user's sign language. This adds customization functions according to the speed and style of sign language movements.

[0075] The talk language enhancement unit can use the emotion estimation function to select an appropriate expression according to the user's emotion, thereby improving the quality of communication. The talk language enhancement unit, for example, uses the emotion estimation function to select an appropriate expression according to the user's emotion. For example, if the user is happy, a positive expression is selected. The emotion estimation function is also used to analyze the user's emotion and provide an appropriate expression. For example, if the user is sad, a kind expression is selected. The emotion estimation function is also used to select an expression according to the user's emotion, thereby improving the quality of communication. For example, if the user is angry, a calm expression is selected. This allows an appropriate expression to be selected according to the user's emotion, thereby improving the quality of communication.

[0076] The communication facilitation unit can add a function that uses the generation AI to translate non-verbal communication between deaf and hearing people. For example, the generation AI analyzes facial expressions and gestures to translate non-verbal communication. For example, it recognizes smiles and hand movements and provides an appropriate translation. The generation AI also analyzes the user's non-verbal communication in real time and reflects this in the translation results. For example, it provides translations that match the user's facial expressions. The generation AI also builds a database of non-verbal communication to improve translation accuracy. For example, it learns different gestures and facial expressions and translates them appropriately. This allows non-verbal communication between deaf and hearing people to be translated.

[0077] The communication facilitation unit can use the generation AI to learn the user's communication history and suggest appropriate responses based on the content of past conversations. For example, the communication facilitation unit uses the generation AI to analyze the user's communication history and suggest appropriate responses based on the content of past conversations. For example, it refers to the content of previous conversations and provides relevant responses. The generation AI also learns the user's communication patterns and automatically generates appropriate responses. For example, it learns the user's speaking style and expressions and provides natural responses. The generation AI also suggests responses tailored to the user's preferences and interests based on past conversation data. For example, it provides responses related to topics that interest the user. In this way, appropriate responses are suggested based on the user's communication history.

[0078] The communication facilitation unit can automatically adjust a communication style according to the user's emotions using the emotion estimation function. The communication facilitation unit, for example, uses the emotion estimation function to automatically adjust a communication style according to the user's emotions. For example, if the user is nervous, the communication facilitation unit responds in a relaxed tone. The emotion estimation function is also used to analyze the user's emotions in real time and provide an appropriate communication style. For example, if the user is happy, the communication facilitation unit responds in a bright tone. The emotion estimation function is also used to provide feedback according to the user's emotions and improve the quality of communication. For example, if the user is sad, the communication facilitation unit responds in a gentle tone. In this way, the communication style according to the user's emotions is automatically adjusted.

[0079] The communication facilitation unit uses generative AI to support communication between deaf and hearing people, not only translating in real time but also understanding the context of the conversation and providing appropriate advice. For example, the generative AI analyzes the context of the conversation and provides appropriate advice. For example, it understands the flow of the conversation and suggests what to say next. The generative AI also understands the context of the conversation while translating in real time and provides appropriate feedback. For example, it provides information related to the topic of the conversation. The generative AI also learns the context of the conversation and provides appropriate advice to the user. For example, it suggests what to say next as the conversation progresses. In this way, understanding the context of the conversation and providing appropriate advice facilitates communication between deaf and hearing people.

[0080] The communication facilitation unit can use the generative AI to add an interactive whiteboard function that can be used jointly by deaf and hearing people. For example, the generative AI provides an interactive whiteboard function that can be used jointly by deaf and hearing people. For example, handwritten notes and drawings can be shared in real time. The generative AI can also translate the content on the whiteboard in real time to enable smooth communication between deaf and hearing people. For example, it can convert handwritten characters into speech. The generative AI can also use the interactive whiteboard function to provide a tool that allows deaf and hearing people to work together on a project. For example, they can jointly brainstorm ideas and share them in real time. This adds an interactive whiteboard function that can be used jointly by deaf and hearing people.

[0081] The device modernization unit uses generation AI to monitor usage status on Defrympook in real time and make immediate improvements. For example, the device modernization unit uses generation AI to monitor usage status on Defrympook in real time and make immediate improvements. For example, it adjusts settings based on user feedback. The generation AI also analyzes usage data on Defrympook and immediately suggests improvements. For example, it prioritizes improvements to frequently used functions. The generation AI also monitors usage status on Defrympook in real time and responds immediately if an abnormality occurs. For example, it detects and corrects system malfunctions. This allows usage status on Defrympook to be monitored in real time and make immediate improvements.

[0082] The device modernization unit can use generation AI to analyze user feedback on Defrimpuk and automatically suggest improvements for the next event. For example, the device modernization unit uses generation AI to analyze user feedback on Defrimpuk and automatically suggest improvements for the next event. For example, new functions are added based on user opinions. The generation AI also collects user feedback data and suggests improvements based on the analysis results. For example, it suggests improvements to resolve user dissatisfaction. The generation AI also analyzes feedback on Defrimpuk in real time and immediately suggests improvements. For example, it adds functions in response to user requests. In this way, user feedback on Defrimpuk is analyzed and improvements for the next event are automatically suggested.

[0083] The device modernization unit can use the emotion estimation function to analyze the user's emotional reactions on DefrinPook and make improvements to provide a positive experience. The device modernization unit, for example, uses the emotion estimation function to analyze the user's emotional reactions on DefrinPook in real time. For example, it analyzes whether the user is enjoying themselves and makes improvements to provide a positive experience. It also uses the emotion estimation function to collect user emotional data and propose improvements based on the analysis results. For example, it improves areas where the user feels dissatisfied. It also uses the emotion estimation function to monitor the user's emotional reactions on DefrinPook and provide feedback to provide a positive experience. For example, it strengthens areas where the user is happy. In this way, it analyzes the user's emotional reactions on DefrinPook and makes improvements to provide a positive experience.

[0084] Based on the success story at DeFrimpuk, the Device Modernization Department can introduce SureTalk at other international events and exhibitions to promote its use. Based on the success story at DeFrimpuk, the Device Modernization Department can introduce SureTalk at other international events and exhibitions to promote its use. For example, they can introduce the success story in a presentation to promote its use. They can also analyze the usage data at DeFrimpuk and develop a strategy for introducing SureTalk at other events. For example, they can analyze the success factors and propose how to apply it at other events. They can also use the success story at DeFrimpuk to promote SureTalk. For example, they can launch a marketing campaign based on the success story. In this way, based on the success story at DeFrimpuk, the Device Modernization Department can introduce SureTalk at other international events and exhibitions to promote its use.

[0085] The device modernization unit can use generative AI to analyze usage data from Defrinpook and develop versions adapted to different markets and cultural spheres. For example, the device modernization unit uses generative AI to analyze usage data from Defrinpook and develop versions adapted to different markets and cultural spheres. For example, functions tailored to specific cultures can be added. The generative AI also collects user data from Defrinpook and develops market-specific versions based on the analysis results. For example, customization can be performed according to regional needs. The generative AI also analyzes usage status on Defrinpook and proposes versions adapted to different markets and cultural spheres. For example, providing designs and functions tailored to specific markets. This makes it possible to analyze usage data from Defrinpook and develop versions adapted to different markets and cultural spheres.

[0086] The device modernization unit can use the emotion estimation function to optimize marketing strategies based on user emotion data on Deflympook and expand global market share. The device modernization unit, for example, uses the emotion estimation function to collect user emotion data on Deflympook and optimize marketing strategies. For example, it can develop campaigns that elicit positive emotions. It can also use the emotion estimation function to analyze users' emotional responses and reflect them in marketing strategies. For example, it can create advertisements that highlight aspects that make users happy. It can also use the emotion estimation function to adjust marketing strategies in real time based on user emotion data on Deflympook. For example, it can carry out promotions in response to changes in users' emotions. In this way, it can optimize marketing strategies based on user emotion data on Deflympook and expand global market share.

[0087] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0088] The SureTalk system can also be equipped with a health management unit that monitors the user's health status. For example, the health management unit can measure the user's heart rate and blood pressure in real time and issue an alert if an abnormality is detected. The health management unit can also analyze the user's sleep patterns and provide advice to promote quality sleep. Furthermore, the health management unit can record the user's exercise volume and propose an exercise plan to maintain health. In this way, the SureTalk system can support the user's health management and provide a more fulfilling life.

[0089] The device updater can also utilize the user's location information to provide an optimal communication environment. For example, it can analyze the communication conditions at the user's location based on the location information and automatically configure the optimal network settings. It can also utilize the location information to make adjustments to maintain stable communication even while the user is on the move. It can also use the location information to provide customized services tailored to the locations the user visits. This improves the device's communication performance and increases user convenience.

[0090] The Talk Language Enhancement Unit can also provide an individually optimized learning plan based on the user's learning history. For example, it can analyze what the user has learned in the past and suggest what they should learn next. It can also monitor the user's learning pace and level of understanding in real time and provide appropriate feedback. It can also customize learning content based on the user's interests. This improves the user's learning efficiency and makes learning more effective.

[0091] The communication facilitation unit can further estimate the user's emotions and suggest an appropriate communication method based on the estimated emotions. For example, if the user is nervous, it can provide advice on how to relax. If the user is angry, it can also suggest ways to calm down. Furthermore, if the user is happy, it can also suggest ways to share those emotions. This promotes communication according to the user's emotions and achieves smoother communication.

[0092] The SureTalk system can also include a content provider that provides customized content based on the user's hobbies and interests. For example, it can automatically collect and provide news and articles related to topics that interest the user. It can also suggest events and activities related to the user's hobbies. It can also provide customized entertainment content based on the user's interests. This allows the user to enjoy a more fulfilling life by receiving content tailored to their interests.

[0093] The device modernization module can further analyze user usage and optimize energy efficiency. For example, it can automatically turn off infrequently used features to reduce battery consumption. It can also suggest energy-efficient settings based on the user's usage patterns. It can also optimize device thermal management to reduce energy consumption. This improves the device's energy efficiency and extends battery life.

[0094] The device updater can further estimate the user's emotions and optimize device operation based on the estimated emotions. For example, it can simplify operation when the user is stressed, or provide detailed settings when the user is relaxed. It can also customize the device interface according to the user's emotions. This allows device operation to be tailored to the user's emotions, providing a more comfortable user experience.

[0095] The talk language enhancement unit can further estimate the user's emotions and provide translations that reflect appropriate tones and nuances based on the estimated emotions. For example, if the user is sad, the translation can be done in a gentle tone. Alternatively, if the user is excited, the translation can be done in an energetic tone. Furthermore, it can add appropriate nuances to the translation results depending on the user's emotions. This provides translations that reflect the tones and nuances according to the user's emotions, realizing more natural communication.

[0096] The SureTalk system can also include a feedback unit that estimates the user's emotions and provides appropriate feedback based on the estimated emotions. For example, if the user is feeling down, it can display an encouraging message. If the user is feeling happy, it can also suggest ways to share those emotions. It can also suggest appropriate actions depending on the user's emotions. This provides feedback according to the user's emotions, resulting in more fulfilling communication.

[0097] The SureTalk system can also be equipped with a reminder section that analyzes the user's behavioral patterns and provides appropriate reminders. For example, if a user often performs a specific action at a specific time, a reminder to encourage that action can be displayed. It can also remind the user of important appointments based on the user's schedule. It can also provide reminders for health management and task management based on the user's behavioral patterns. This provides reminders that are tailored to the user's behavioral patterns, leading to a more efficient lifestyle.

[0098] The processing flow of the second embodiment will be briefly explained below.

[0099] Step 1: The device modernization unit uses generative AI to modernize the device. For example, generative AI can optimize the placement of sensors and microphones on the device. Generative AI can also optimize the device to extend its battery life. Generative AI can also design the device to be lightweight. Step 2: The talk language expansion unit uses the generation AI to expand the talk language. For example, the generation AI analyzes prompts entered by the user and generates an operating program that translates them into an appropriate language or sign language. The generation AI can also generate programs that enable multilingual support. Furthermore, the generation AI can generate programs that recognize and translate sign language and gestures. Step 3: The communication facilitation unit uses the generation AI to facilitate communication between deaf and hearing people. For example, the generation AI can convert what a deaf person says in sign language into audio in real time and convey it to a hearing person. The generation AI can also convert what a hearing person says into sign language in real time and convey it to a deaf person. Furthermore, the generation AI can generate programs to facilitate smooth communication between deaf and hearing people.

[0100] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0101] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0102] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0103] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0104] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0105] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0106] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0107] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0108] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0109] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0110] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0111] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0112] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0113] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0114] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0115] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0116] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0117] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0118] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0119] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0120] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0121] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0122] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0123] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0124] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0125] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0126] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0127] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0128] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 may also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0129] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0130] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0131] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0132] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0133] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0134] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0135] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0136] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0137] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0138] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0139] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0140] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0141] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0142] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0143] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0144] In the robot 414, the processor 46 performs the identification process. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0145] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0146] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0147] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0148] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0149] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0150] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0151] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0152] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0153] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[0154] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0155] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0156] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0157] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0158] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0159] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0160] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0161] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0162] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0163] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0164] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0165] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0166] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0167] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a device update unit that updates the device using a generation AI; a talk language expansion unit that expands the talk language using generative AI; and a communication facilitation unit that uses generation AI to facilitate communication between deaf and hearing people. A system characterized by:

2. The device updater includes: Add an environmental sensor to the device to automatically adjust the sensitivity of voice recognition according to the ambient noise level.

2. The system of claim 1.

3. The talk language expansion unit Using the generative AI, the talk language will be expanded to accommodate regional dialects and slang.

2. The system of claim 1.

4. The communication facilitation unit Add a function to translate non-verbal communication between the deaf and hearing people using the generative AI.

2. The system of claim 1.

5. The device updater includes: Detecting a user's stress level and simplifying the operation of the device 2. The system of claim 1.

Images