System

Abstract

To provide a system that smoothly facilitates communication with people who are hearing impaired or have other disabilities.SOLUTION: A system according to the embodiment comprises a voice recognition unit, a conversion unit, and a subtitle display unit. The voice recognition unit recognizes voice. The conversion unit converts the voice recognized by the voice recognition unit into text. The subtitle display unit displays the text converted by the conversion unit on a wearable device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has the problem that it is difficult to communicate smoothly with people who are hearing impaired or have other disabilities.

[0005] The system according to the embodiment aims to facilitate communication with people who have hearing impairments or disabilities. [Means for solving the problem]

[0006] The system according to the embodiment includes a voice recognition unit, a conversion unit, and a subtitle display unit. The voice recognition unit recognizes voice. The conversion unit converts the voice recognized by the voice recognition unit into text. The subtitle display unit displays the text converted by the conversion unit on the wearable device. Effect of the Invention

[0007] The system according to the embodiment can facilitate communication with people who are hearing impaired or have other disabilities. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A system according to an embodiment of the present invention is a device and a service for improving communication with people with hearing impairments or disabilities. This system displays subtitles of what the person in front of you says within the field of view through glasses. The system aims to improve accuracy by adding lip reading to voice recognition, and learns face authentication and voiceprint data to help with identification from the second time onwards. In addition, by displaying other sound information, it is possible to grasp sounds from invisible places, which contributes to safety when walking, etc. For example, in the system, the voice recognition unit recognizes voice, and the conversion unit converts the voice into text. Next, the subtitle display unit displays the converted text on a wearable device (lens of glasses). Furthermore, the face authentication unit authenticates the face of the person who speaks the voice, and the lip reading unit performs lip reading according to the identified person. This improves the accuracy of voice recognition. The conversion unit also converts environmental sound information into text, and the translation unit provides a translation function. The voice recognition unit recognizes voice in real time, and the subtitle display unit displays the converted text on the lens of glasses. This allows people with hearing impairments or disabilities to visually grasp what is being said by the person in front of them and the surrounding sound information.This allows people with hearing impairments or disabilities to visually grasp what is being said by the person in front of them and the surrounding sound information.

[0029] The communication support system according to the embodiment includes a voice recognition unit, a conversion unit, and a subtitle display unit. The voice recognition unit recognizes voice. The voice recognition unit recognizes voice using, for example, an acoustic model, a language model, or a deep learning technology. The voice recognition unit can also recognize voice using, for example, a voice recognition engine. The voice recognition unit can also recognize voice using, for example, a natural language processing technology. The conversion unit converts the voice recognized by the voice recognition unit into text. The conversion unit converts the voice into text using, for example, a voice recognition engine. The conversion unit can also convert the voice into text using, for example, a natural language processing technology. The conversion unit can also convert the voice into text using, for example, a voice recognition engine. The subtitle display unit displays the text converted by the conversion unit on a wearable device. The subtitle display unit displays the text on a wearable device such as, for example, smart glasses or a smart watch. The subtitle display unit can also display the text using, for example, an AR technology. The subtitle display unit can also display the text using, for example, a display technology. As a result, the communication support system according to the embodiment enables a person with a hearing impairment or disability to visually grasp what is being said by the person in front of them and surrounding sound information.

[0030] The voice recognition unit recognizes voice. The voice recognition unit recognizes voice using, for example, an acoustic model, a language model, and deep learning technology. Specifically, the acoustic model analyzes the voice signal and breaks it down into basic units such as phonemes and syllables. The language model combines these phonemes and syllables to convert them into meaningful words and phrases. The deep learning technology learns a large amount of voice data and improves the accuracy of voice recognition. The voice recognition unit can also recognize voice using, for example, a voice recognition engine. The voice recognition engine converts the voice signal into digital data and performs analysis in real time. As a result, the voice recognition unit learns the characteristics of the speaker's voice and pronunciation habits, achieving more accurate voice recognition. The voice recognition unit can also recognize voice using, for example, natural language processing technology. Natural language processing technology can not only convert voice data into text data, but also understand the context and meaning and generate an appropriate response. As a result, the voice recognition unit can provide more advanced communication support than simply converting voice into text. Furthermore, the voice recognition unit can remove surrounding noise using noise canceling technology to obtain clear voice data. This enables highly accurate voice recognition even in noisy environments. The voice recognition unit can also use multiple microphones to localize the sound source and pick up the voice of a specific speaker. This makes it possible to accurately recognize the voice of a specific speaker even when multiple speakers are speaking at the same time.

[0031] The conversion unit converts the voice recognized by the voice recognition unit into text. The conversion unit converts the voice into text, for example, using a voice recognition engine. The voice recognition engine analyzes the voice signal and generates corresponding text data. Specifically, the corresponding text is generated by extracting features of the voice signal and collating the features with an existing voice database. The conversion unit can also convert the voice into text, for example, using natural language processing technology. Natural language processing technology generates more natural and accurate text by analyzing voice data and understanding the context and meaning. This allows the conversion unit to generate appropriate text according to the context, rather than simply converting the voice into text. The conversion unit can also convert the voice into text, for example, using a voice recognition engine. The voice recognition engine analyzes the voice signal in real time and instantly generates text data. This allows the conversion unit to convert the voice into text in real time and instantly display it. Furthermore, the conversion unit can learn the user's pronunciation and speaking characteristics in order to improve the accuracy of the voice recognition. This allows the conversion unit to provide voice recognition and text conversion optimized for each user. The conversion unit can also support multiple languages ​​and can convert speech in different languages ​​into text appropriately, enabling the conversion unit to achieve highly accurate speech recognition and text conversion even in a multilingual environment.

[0032] The subtitle display unit displays the text converted by the conversion unit on the wearable device. The subtitle display unit displays the text on a wearable device such as, for example, smart glasses or a smart watch. The smart glasses can display text directly in the user's field of vision, allowing the user to check information without moving their line of sight. The smart watch can be worn on the wrist, allowing the user to easily check text information. The subtitle display unit can also display text using, for example, AR technology. The AR technology displays text information superimposed on the real field of vision, allowing the user to check information in a natural manner. This allows the subtitle display unit to reduce the visual burden on the user and provide comfortable information provision. The subtitle display unit can also display text using, for example, display technology. The display technology realizes clear text display with high resolution and improves visibility. This allows the user to easily read the text information on the subtitle display unit. Furthermore, the subtitle display unit can customize the font size, color, and background color of the text to be displayed, allowing the user to provide a display according to the user's visual preferences. The subtitle display unit can display text information that is updated in real time, and can provide the latest information instantly in cooperation with the speech recognition unit and conversion unit. This allows the user to always be aware of the latest information, and effectively supports communication.

[0033] The face authentication unit authenticates the face of the person who speaks the voice, and the lip reading unit can perform lip reading according to the person specified by the face authentication unit. The face authentication unit authenticates the face of the person who speaks the voice, for example, using a face recognition algorithm. The face authentication unit can also authenticate the face of the person who speaks the voice, for example, using an image processing technology. The face authentication unit can also authenticate the face of the person who speaks the voice, for example, using a face recognition algorithm. The lip reading unit, for example, analyzes the movement of the lips and performs lip reading according to the specified person. The lip reading unit can also analyze the movement of the lips, for example, using a machine learning technology, and perform lip reading according to the specified person. The lip reading unit can also analyze the movement of the lips and perform lip reading according to the specified person. This improves the accuracy of voice recognition by face authentication and lip reading. A part or all of the above-mentioned processing in the face authentication unit and the lip reading unit may be performed, for example, using AI, or may be performed without using AI. For example, the face authentication unit can use a face recognition algorithm to authenticate the face of a person speaking, and the lip reading unit can use machine learning technology to analyze lip movements and perform lip reading according to the identified person.

[0034] The conversion unit can convert the information on the environmental sound into text. The conversion unit converts the information on the environmental sound, such as background sound, noise, and specific sound, into text. The conversion unit can also convert the information on the environmental sound into text, for example, using a voice recognition engine. The conversion unit can also convert the information on the environmental sound into text, for example, using natural language processing technology. The conversion unit can also convert the information on the environmental sound into text, for example, using a voice recognition engine. As a result, by converting the environmental sound into text, it is possible to grasp sounds from invisible places. A part or all of the above-mentioned processing in the conversion unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversion unit can convert the information on the environmental sound into text using a voice recognition engine, so that sounds from invisible places can be grasped.

[0035] The translation unit can provide a translation function. The translation unit can provide the translation function using, for example, machine translation. The translation unit can also provide the translation function using, for example, natural language processing technology. The translation unit can also provide the translation function using, for example, machine translation. This allows the translation function to understand speech in different languages. Some or all of the above-mentioned processing in the translation unit can be performed using, for example, AI, or can be performed without using AI. For example, the translation unit can provide the translation function using machine translation, making it possible to understand speech in different languages.

[0036] The voice recognition unit can recognize voice in real time. The voice recognition unit can instantly recognize voice using, for example, real-time processing. The voice recognition unit can also instantly recognize voice using, for example, low-latency technology. The voice recognition unit can also instantly recognize voice using, for example, real-time processing. This allows for instant subtitle display by recognizing voice in real time. A part or all of the above-mentioned processing in the voice recognition unit can be performed using, for example, AI, or can be performed without using AI. For example, the voice recognition unit can instantly recognize voice using real-time processing, allowing instant subtitle display.

[0037] The subtitle display unit can display the converted text on the lens of the glasses. The subtitle display unit can display the converted text on the lens of the glasses, for example, using AR technology. The subtitle display unit can also display the converted text on the lens of the glasses, for example, using display technology. The subtitle display unit can also display the converted text on the lens of the glasses, for example, using AR technology. This allows information to be visually grasped by displaying it on the lens of the glasses. A part or all of the above-mentioned processing in the subtitle display unit can be performed, for example, using AI, or can be performed without using AI. For example, the subtitle display unit can display the converted text on the lens of the glasses, using AR technology, to allow information to be visually grasped.

[0038] The voice recognition unit can automatically filter background noise to improve the recognition accuracy. The voice recognition unit can automatically filter background noise using, for example, a noise canceling technology to improve the recognition accuracy. The voice recognition unit can also automatically filter background noise using, for example, a signal processing technology to improve the recognition accuracy. The voice recognition unit can also filter noise in a specific frequency band to improve the recognition accuracy. As a result, the accuracy of voice recognition is improved by filtering the background noise. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can automatically filter background noise using a noise canceling technology to improve the recognition accuracy.

[0039] The voice recognition unit can dynamically adjust the recognition algorithm according to the speaking speed of the speaker. The voice recognition unit adjusts the recognition algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. The voice recognition unit can also adjust the recognition algorithm according to the speaking speed of the speaker using, for example, adaptive filtering. The voice recognition unit can also adjust the recognition algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. As a result, the recognition accuracy is improved by adjusting the recognition algorithm according to the speaking speed of the speaker. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can adjust the recognition algorithm according to the speaking speed of the speaker using a dynamic adjustment algorithm to improve the recognition accuracy.

[0040] The voice recognition unit can improve the recognition accuracy by taking into account the geographical accent of the speaker. The voice recognition unit can improve the recognition accuracy by taking into account the geographical accent of the speaker using, for example, a dialect model. The voice recognition unit can also improve the recognition accuracy by taking into account the geographical accent of the speaker using, for example, region-specific voice recognition. The voice recognition unit can also improve the recognition accuracy by taking into account the geographical accent of the speaker using, for example, a dialect model. As a result, the recognition accuracy is improved by taking into account the geographical accent of the speaker. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can improve the recognition accuracy by taking into account the geographical accent of the speaker using a dialect model.

[0041] The voice recognition unit can improve the recognition accuracy by referring to the speaker's past speech history. The voice recognition unit can improve the recognition accuracy by referring to the speaker's past speech history using, for example, a history database. The voice recognition unit can also improve the recognition accuracy by referring to the speaker's past speech history using, for example, a history analysis algorithm. The voice recognition unit can also improve the recognition accuracy by referring to the speaker's past speech history using, for example, a history database. As a result, the recognition accuracy is improved by referring to the speaker's past speech history. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can improve the recognition accuracy by referring to the speaker's past speech history using a history database.

[0042] The conversion unit can analyze the context of the voice and convert it into more natural text. The conversion unit can analyze the context of the voice using, for example, natural language processing technology and convert it into more natural text. The conversion unit can also analyze the context of the voice using, for example, a context model and convert it into more natural text. The conversion unit can also analyze the context of the voice using, for example, natural language processing technology and convert it into more natural text. As a result, more natural text is generated by analyzing the context of the voice. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can analyze the context of the voice using natural language processing technology and convert it into more natural text.

[0043] The conversion unit can automatically recognize technical terms and slang and convert them into appropriate text. The conversion unit can automatically recognize technical terms using, for example, a technical dictionary and convert them into appropriate text. The conversion unit can also automatically recognize slang using, for example, a slang database and convert them into appropriate text. The conversion unit can automatically recognize technical terms using, for example, a technical dictionary and convert them into appropriate text. As a result, more understandable text is generated by appropriately converting technical terms and slang. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can automatically recognize technical terms using a technical dictionary and convert them into appropriate text.

[0044] The conversion unit can analyze the intention of the speaker of the voice and convert it into appropriate text. The conversion unit can analyze the intention of the speaker of the voice using, for example, an intention recognition algorithm and convert it into appropriate text. The conversion unit can also analyze the intention of the speaker of the voice using, for example, a natural language processing technology and convert it into appropriate text. The conversion unit can also analyze the intention of the speaker of the voice using, for example, an intention recognition algorithm and convert it into appropriate text. In this way, more appropriate text is generated by analyzing the intention of the speaker. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can analyze the intention of the speaker of the voice using an intention recognition algorithm and convert it into appropriate text.

[0045] The conversion unit can convert the speech into text while taking into account background information of the speech. The conversion unit can convert the speech into text while taking into account background information of the speech, for example, using context analysis. The conversion unit can also convert the speech into text while taking into account background information of the speech, for example, using extraction of related information. The conversion unit can also convert the speech into text while taking into account background information of the speech, for example, using context analysis. As a result, more appropriate text is generated by taking into account the background information of the speech. A part or all of the above-mentioned processing in the conversion unit can be performed, for example, using AI, or can be performed without using AI. For example, the conversion unit can convert the speech into text while taking into account background information of the speech, using context analysis.

[0046] The subtitle display unit can track the user's gaze and determine an appropriate display position. The subtitle display unit can track the user's gaze using, for example, an eye-tracking sensor, and determine an appropriate display position. The subtitle display unit can also track the user's gaze using, for example, an eye-tracking analysis algorithm, and determine an appropriate display position. The subtitle display unit can track the user's gaze using, for example, an eye-tracking sensor, and determine an appropriate display position. As a result, by tracking the user's gaze, subtitles with high visibility are displayed. A part or all of the above-mentioned processing in the subtitle display unit may be performed using, for example, AI, or may be performed without using AI. For example, the subtitle display unit can track the user's gaze using an eye-tracking sensor, and determine an appropriate display position.

[0047] The subtitle display unit can dynamically adjust the font and size of the text to be displayed. The subtitle display unit can adjust the font and size of the text to be displayed, for example, by using a dynamic layout engine. The subtitle display unit can also adjust the font and size of the text to be displayed, for example, by using adaptive font size adjustment. The subtitle display unit can also adjust the font and size of the text to be displayed, for example, by using a dynamic layout engine. This improves visibility by dynamically adjusting the font and size. A part or all of the above-mentioned processing in the subtitle display unit can be performed, for example, by using AI or without using AI. For example, the subtitle display unit can adjust the font and size of the text to be displayed, by using a dynamic layout engine.

[0048] The subtitle display unit can adopt a display method for reducing visual fatigue of the user. The subtitle display unit can reduce visual fatigue, for example, by using blue light cut. The subtitle display unit can also reduce visual fatigue, for example, by using adaptive brightness adjustment. The subtitle display unit can also reduce visual fatigue, for example, by using blue light cut. This reduces visual fatigue, making it possible to use the device for a long time. A part or all of the above-mentioned processing in the subtitle display unit can be performed, for example, by using AI, or can be performed without using AI. For example, the subtitle display unit can reduce visual fatigue by using blue light cut.

[0049] The subtitle display unit can adjust the brightness of the display in consideration of the user's ambient light. The subtitle display unit can adjust the brightness of the display in consideration of the user's ambient light, for example, using a light sensor. The subtitle display unit can also adjust the brightness of the display in consideration of the user's ambient light, for example, using adaptive brightness adjustment. The subtitle display unit can also adjust the brightness of the display in consideration of the user's ambient light, for example, using a light sensor. Thereby, the visibility is improved by adjusting the brightness of the display according to the ambient light. A part or all of the above-mentioned processing in the subtitle display unit may be performed, for example, using AI, or may be performed without using AI. For example, the subtitle display unit can adjust the brightness of the display in consideration of the user's ambient light, using a light sensor.

[0050] The face authentication unit can improve the authentication accuracy by taking into account changes in facial expression. The face authentication unit can improve the authentication accuracy by taking into account changes in facial expression using, for example, a facial expression recognition algorithm. The face authentication unit can also improve the authentication accuracy by taking into account changes in facial expression using, for example, dynamic facial expression analysis. The face authentication unit can also improve the authentication accuracy by taking into account changes in facial expression using, for example, a facial expression recognition algorithm. As a result, the authentication accuracy is improved by taking into account changes in facial expression. A part or all of the above-mentioned processing in the face authentication unit may be performed using, for example, AI, or may be performed without using AI. For example, the face authentication unit can improve the authentication accuracy by taking into account changes in facial expression using a facial expression recognition algorithm.

[0051] The face authentication unit can recognize and identify multiple faces simultaneously. The face authentication unit can recognize and identify multiple faces simultaneously, for example, using a multi-face recognition algorithm. The face authentication unit can also recognize and identify multiple faces simultaneously, for example, using a parallel processing technique. The face authentication unit can also recognize and identify multiple faces simultaneously, for example, using a multi-face recognition algorithm. This improves authentication efficiency by recognizing multiple faces simultaneously. A part or all of the above-mentioned processing in the face authentication unit may be performed, for example, using AI, or may be performed without using AI. For example, the face authentication unit can recognize and identify multiple faces simultaneously, using a multi-face recognition algorithm.

[0052] The face authentication unit can improve the authentication accuracy by taking into account the angle of the face and the lighting conditions. The face authentication unit can improve the authentication accuracy by taking into account the angle of the face using, for example, 3D face recognition. The face authentication unit can also improve the authentication accuracy by taking into account the lighting conditions using, for example, a lighting correction algorithm. The face authentication unit can also improve the authentication accuracy by taking into account the angle of the face using, for example, 3D face recognition. As a result, the authentication accuracy is improved by taking into account the angle of the face and the lighting conditions. A part or all of the above-mentioned processing in the face authentication unit may be performed using, for example, AI, or may be performed without using AI. For example, the face authentication unit can improve the authentication accuracy by taking into account the angle of the face using 3D face recognition.

[0053] The face authentication unit can improve the authentication accuracy by referring to the past authentication history. The face authentication unit can improve the authentication accuracy by referring to the past authentication history, for example, using a history database. The face authentication unit can also improve the authentication accuracy by referring to the past authentication history, for example, using a history analysis algorithm. The face authentication unit can also improve the authentication accuracy by referring to the past authentication history, for example, using a history database. As a result, the authentication accuracy is improved by referring to the past authentication history. A part or all of the above-mentioned processing in the face authentication unit may be performed, for example, using AI, or may be performed without using AI. For example, the face authentication unit can improve the authentication accuracy by referring to the past authentication history, using a history database.

[0054] The lip reading unit can analyze the movement of the speaker's lips at high resolution to improve accuracy. The lip reading unit can analyze the movement of the speaker's lips at high resolution, for example, using a high resolution camera. The lip reading unit can also analyze the movement of the speaker's lips at high resolution, for example, using an image processing algorithm. The lip reading unit can also analyze the movement of the speaker's lips at high resolution, for example, using a high resolution camera. This improves the accuracy of lip reading by analyzing the movement of the lips at high resolution. A part or all of the above-mentioned processing in the lip reading unit can be performed, for example, using AI, or can be performed without using AI. For example, the lip reading unit can analyze the movement of the speaker's lips at high resolution using a high resolution camera to improve the accuracy of lip reading.

[0055] The lip reading unit can dynamically adjust the analysis algorithm according to the speaking speed of the speaker. The lip reading unit can adjust the analysis algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. The lip reading unit can also adjust the analysis algorithm according to the speaking speed of the speaker using, for example, adaptive filtering. The lip reading unit can also adjust the analysis algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. Thereby, by adjusting the analysis algorithm according to the speaking speed, the accuracy of lip reading is improved. A part or all of the above-mentioned processing in the lip reading unit can be performed using, for example, AI, or can be performed without using AI. For example, the lip reading unit can adjust the analysis algorithm according to the speaking speed of the speaker using a dynamic adjustment algorithm to improve the accuracy of lip reading.

[0056] The lip reading unit can improve the analysis accuracy by taking into account the change in the speaker's facial expression. The lip reading unit can improve the analysis accuracy by taking into account the change in the speaker's facial expression using, for example, a facial expression recognition algorithm. The lip reading unit can also improve the analysis accuracy by taking into account the change in the speaker's facial expression using, for example, a dynamic facial expression analysis. The lip reading unit can also improve the analysis accuracy by taking into account the change in the speaker's facial expression using, for example, a facial expression recognition algorithm. Thereby, the accuracy of lip reading is improved by taking into account the change in the facial expression. A part or all of the above-mentioned processing in the lip reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the lip reading unit can improve the analysis accuracy by taking into account the change in the speaker's facial expression using a facial expression recognition algorithm.

[0057] The lip reading unit can improve the analysis accuracy by referring to the speaker's past speech history. The lip reading unit can improve the analysis accuracy by referring to the speaker's past speech history using, for example, a history database. The lip reading unit can also improve the analysis accuracy by referring to the speaker's past speech history using, for example, a history analysis algorithm. The lip reading unit can also improve the analysis accuracy by referring to the speaker's past speech history using, for example, a history database. As a result, the accuracy of lip reading is improved by referring to the past speech history. A part or all of the above-mentioned processing in the lip reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the lip reading unit can improve the analysis accuracy by referring to the speaker's past speech history using a history database.

[0058] The translation unit can analyze the context to provide a more natural translation. The translation unit can analyze the context using, for example, natural language processing technology to provide a more natural translation. The translation unit can also analyze the context using, for example, a context model to provide a more natural translation. The translation unit can also analyze the context using, for example, natural language processing technology to provide a more natural translation. Thereby, a more natural translation is provided by analyzing the context. A part or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without using AI. For example, the translation unit can analyze the context using natural language processing technology to provide a more natural translation.

[0059] The translation unit can automatically recognize technical terms and slang and provide an appropriate translation. The translation unit can automatically recognize technical terms using, for example, a technical dictionary and provide an appropriate translation. The translation unit can also automatically recognize slang using, for example, a slang database and provide an appropriate translation. The translation unit can automatically recognize technical terms using, for example, a technical dictionary and provide an appropriate translation. As a result, a translation that is easier to understand is provided by appropriately translating technical terms and slang. Some or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without using AI. For example, the translation unit can automatically recognize technical terms using a technical dictionary and provide an appropriate translation.

[0060] The translation unit can analyze the intention of the speaker and provide an appropriate translation. The translation unit can analyze the intention of the speaker using, for example, an intention recognition algorithm, and provide an appropriate translation. The translation unit can also analyze the intention of the speaker using, for example, a natural language processing technique, and provide an appropriate translation. The translation unit can also analyze the intention of the speaker using, for example, an intention recognition algorithm, and provide an appropriate translation. In this way, a more appropriate translation is provided by analyzing the intention of the speaker. A part or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without using AI. For example, the translation unit can analyze the intention of the speaker using an intention recognition algorithm, and provide an appropriate translation.

[0061] The translation unit can provide a translation taking into account background information of the speaker. The translation unit can provide a translation taking into account background information of the speaker, for example, using context analysis. The translation unit can also provide a translation taking into account background information of the speaker, for example, using related information extraction. The translation unit can also provide a translation taking into account background information of the speaker, for example, using context analysis. Thereby, by taking into account background information of the speaker, a more appropriate translation is provided. Some or all of the above-mentioned processing in the translation unit can be performed, for example, using AI, or can be performed without using AI. For example, the translation unit can provide a translation taking into account background information of the speaker, using context analysis.

[0062] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0063] The speech recognition unit can improve the recognition accuracy by taking into account the geographic accent of the speaker. For example, the speech recognition unit improves the recognition accuracy by taking into account the geographic accent of the speaker using a dialect model. The speech recognition unit can also improve the recognition accuracy by taking into account the geographic accent of the speaker using region-specific speech recognition. The speech recognition unit can also improve the recognition accuracy by taking into account the geographic accent of the speaker using a dialect model. As a result, the recognition accuracy is improved by taking into account the geographic accent of the speaker.

[0064] The subtitle display unit can track the user's gaze and determine an appropriate display position. For example, the subtitle display unit uses an eye-tracking sensor to track the user's gaze and determine an appropriate display position. The subtitle display unit can also use an eye-tracking analysis algorithm to track the user's gaze and determine an appropriate display position. The subtitle display unit can also use an eye-tracking sensor to track the user's gaze and determine an appropriate display position. In this way, highly visible subtitles are displayed by tracking the user's gaze.

[0065] The face authentication unit can recognize and identify multiple faces simultaneously. For example, the face authentication unit uses a multi-face recognition algorithm to recognize and identify multiple faces simultaneously. The face authentication unit can also use a parallel processing technique to recognize and identify multiple faces simultaneously. The face authentication unit can also use a multi-face recognition algorithm to recognize and identify multiple faces simultaneously. This improves authentication efficiency by recognizing multiple faces simultaneously.

[0066] The lip reading unit may analyze the speaker's lip movements at high resolution to improve accuracy. For example, the lip reading unit may use a high resolution camera to analyze the speaker's lip movements at high resolution. The lip reading unit may also use an image processing algorithm to analyze the speaker's lip movements at high resolution. The lip reading unit may also use a high resolution camera to analyze the speaker's lip movements at high resolution. Thus, by analyzing the lip movements at high resolution, the accuracy of lip reading is improved.

[0067] The process flow of the first embodiment will be briefly described below.

[0068] Step 1: The speech recognition unit recognizes speech. The speech recognition unit can recognize speech using, for example, an acoustic model, a language model, deep learning technology, a speech recognition engine, or natural language processing technology. Step 2: The conversion unit converts the voice recognized by the voice recognition unit into text. The conversion unit can convert the voice into text using, for example, a voice recognition engine or natural language processing technology. Step 3: The subtitle display unit displays the text converted by the conversion unit on the wearable device. The subtitle display unit displays the text on a wearable device such as smart glasses or a smart watch. In addition, the text can be displayed using AR technology or display technology.

[0069] (Example 2) A system according to an embodiment of the present invention is a device and a service for improving communication with people with hearing impairments or disabilities. This system displays subtitles of what the person in front of you says within the field of view through glasses. The system aims to improve accuracy by adding lip reading to voice recognition, and learns face authentication and voiceprint data to help with identification from the second time onwards. In addition, by displaying other sound information, it is possible to grasp sounds from invisible places, which contributes to safety when walking, etc. For example, in the system, the voice recognition unit recognizes voice, and the conversion unit converts the voice into text. Next, the subtitle display unit displays the converted text on a wearable device (lens of glasses). Furthermore, the face authentication unit authenticates the face of the person who speaks the voice, and the lip reading unit performs lip reading according to the identified person. This improves the accuracy of voice recognition. The conversion unit also converts environmental sound information into text, and the translation unit provides a translation function. The voice recognition unit recognizes voice in real time, and the subtitle display unit displays the converted text on the lens of glasses. This allows people with hearing impairments or disabilities to visually grasp what is being said by the person in front of them and the surrounding sound information.This allows people with hearing impairments or disabilities to visually grasp what is being said by the person in front of them and the surrounding sound information.

[0070] The communication support system according to the embodiment includes a voice recognition unit, a conversion unit, and a subtitle display unit. The voice recognition unit recognizes voice. The voice recognition unit recognizes voice using, for example, an acoustic model, a language model, or a deep learning technology. The voice recognition unit can also recognize voice using, for example, a voice recognition engine. The voice recognition unit can also recognize voice using, for example, a natural language processing technology. The conversion unit converts the voice recognized by the voice recognition unit into text. The conversion unit converts the voice into text using, for example, a voice recognition engine. The conversion unit can also convert the voice into text using, for example, a natural language processing technology. The conversion unit can also convert the voice into text using, for example, a voice recognition engine. The subtitle display unit displays the text converted by the conversion unit on a wearable device. The subtitle display unit displays the text on a wearable device such as, for example, smart glasses or a smart watch. The subtitle display unit can also display the text using, for example, an AR technology. The subtitle display unit can also display the text using, for example, a display technology. As a result, the communication support system according to the embodiment enables a person with a hearing impairment or disability to visually grasp what is being said by the person in front of them and surrounding sound information.

[0071] The voice recognition unit recognizes voice. The voice recognition unit recognizes voice using, for example, an acoustic model, a language model, and deep learning technology. Specifically, the acoustic model analyzes the voice signal and breaks it down into basic units such as phonemes and syllables. The language model combines these phonemes and syllables to convert them into meaningful words and phrases. The deep learning technology learns a large amount of voice data and improves the accuracy of voice recognition. The voice recognition unit can also recognize voice using, for example, a voice recognition engine. The voice recognition engine converts the voice signal into digital data and performs analysis in real time. As a result, the voice recognition unit learns the characteristics of the speaker's voice and pronunciation habits, achieving more accurate voice recognition. The voice recognition unit can also recognize voice using, for example, natural language processing technology. Natural language processing technology can not only convert voice data into text data, but also understand the context and meaning and generate an appropriate response. As a result, the voice recognition unit can provide more advanced communication support than simply converting voice into text. Furthermore, the voice recognition unit can remove surrounding noise using noise canceling technology to obtain clear voice data. This enables highly accurate voice recognition even in noisy environments. The voice recognition unit can also use multiple microphones to localize the sound source and pick up the voice of a specific speaker. This makes it possible to accurately recognize the voice of a specific speaker even when multiple speakers are speaking at the same time.

[0072] The conversion unit converts the voice recognized by the voice recognition unit into text. The conversion unit converts the voice into text, for example, using a voice recognition engine. The voice recognition engine analyzes the voice signal and generates corresponding text data. Specifically, the corresponding text is generated by extracting features of the voice signal and collating the features with an existing voice database. The conversion unit can also convert the voice into text, for example, using natural language processing technology. Natural language processing technology generates more natural and accurate text by analyzing voice data and understanding the context and meaning. This allows the conversion unit to generate appropriate text according to the context, rather than simply converting the voice into text. The conversion unit can also convert the voice into text, for example, using a voice recognition engine. The voice recognition engine analyzes the voice signal in real time and instantly generates text data. This allows the conversion unit to convert the voice into text in real time and instantly display it. Furthermore, the conversion unit can learn the user's pronunciation and speaking characteristics in order to improve the accuracy of the voice recognition. This allows the conversion unit to provide voice recognition and text conversion optimized for each user. The conversion unit can also support multiple languages ​​and can convert speech in different languages ​​into text appropriately, enabling the conversion unit to achieve highly accurate speech recognition and text conversion even in a multilingual environment.

[0073] The subtitle display unit displays the text converted by the conversion unit on the wearable device. The subtitle display unit displays the text on a wearable device such as, for example, smart glasses or a smart watch. The smart glasses can display text directly in the user's field of vision, allowing the user to check information without moving their line of sight. The smart watch can be worn on the wrist, allowing the user to easily check text information. The subtitle display unit can also display text using, for example, AR technology. The AR technology displays text information superimposed on the real field of vision, allowing the user to check information in a natural manner. This allows the subtitle display unit to reduce the visual burden on the user and provide comfortable information provision. The subtitle display unit can also display text using, for example, display technology. The display technology realizes clear text display with high resolution and improves visibility. This allows the user to easily read the text information on the subtitle display unit. Furthermore, the subtitle display unit can customize the font size, color, and background color of the text to be displayed, allowing the user to provide a display according to the user's visual preferences. The subtitle display unit can display text information that is updated in real time, and can provide the latest information instantly in cooperation with the speech recognition unit and conversion unit. This allows the user to always be aware of the latest information, and effectively supports communication.

[0074] The face authentication unit authenticates the face of the person who speaks the voice, and the lip reading unit can perform lip reading according to the person specified by the face authentication unit. The face authentication unit authenticates the face of the person who speaks the voice, for example, using a face recognition algorithm. The face authentication unit can also authenticate the face of the person who speaks the voice, for example, using an image processing technology. The face authentication unit can also authenticate the face of the person who speaks the voice, for example, using a face recognition algorithm. The lip reading unit, for example, analyzes the movement of the lips and performs lip reading according to the specified person. The lip reading unit can also analyze the movement of the lips, for example, using a machine learning technology, and perform lip reading according to the specified person. The lip reading unit can also analyze the movement of the lips and perform lip reading according to the specified person. This improves the accuracy of voice recognition by face authentication and lip reading. A part or all of the above-mentioned processing in the face authentication unit and the lip reading unit may be performed, for example, using AI, or may be performed without using AI. For example, the face authentication unit can use a face recognition algorithm to authenticate the face of a person speaking, and the lip reading unit can use machine learning technology to analyze lip movements and perform lip reading according to the identified person.

[0075] The conversion unit can convert the information on the environmental sound into text. The conversion unit converts the information on the environmental sound, such as background sound, noise, and specific sound, into text. The conversion unit can also convert the information on the environmental sound into text, for example, using a voice recognition engine. The conversion unit can also convert the information on the environmental sound into text, for example, using natural language processing technology. The conversion unit can also convert the information on the environmental sound into text, for example, using a voice recognition engine. As a result, by converting the environmental sound into text, it is possible to grasp sounds from invisible places. A part or all of the above-mentioned processing in the conversion unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversion unit can convert the information on the environmental sound into text using a voice recognition engine, so that sounds from invisible places can be grasped.

[0076] The translation unit can provide a translation function. The translation unit can provide the translation function using, for example, machine translation. The translation unit can also provide the translation function using, for example, natural language processing technology. The translation unit can also provide the translation function using, for example, machine translation. This allows the translation function to understand speech in different languages. Some or all of the above-mentioned processing in the translation unit can be performed using, for example, AI, or can be performed without using AI. For example, the translation unit can provide the translation function using machine translation, making it possible to understand speech in different languages.

[0077] The voice recognition unit can recognize voice in real time. The voice recognition unit can instantly recognize voice using, for example, real-time processing. The voice recognition unit can also instantly recognize voice using, for example, low-latency technology. The voice recognition unit can also instantly recognize voice using, for example, real-time processing. This allows for instant subtitle display by recognizing voice in real time. A part or all of the above-mentioned processing in the voice recognition unit can be performed using, for example, AI, or can be performed without using AI. For example, the voice recognition unit can instantly recognize voice using real-time processing, allowing instant subtitle display.

[0078] The subtitle display unit can display the converted text on the lens of the glasses. The subtitle display unit can display the converted text on the lens of the glasses, for example, using AR technology. The subtitle display unit can also display the converted text on the lens of the glasses, for example, using display technology. The subtitle display unit can also display the converted text on the lens of the glasses, for example, using AR technology. This allows information to be visually grasped by displaying it on the lens of the glasses. A part or all of the above-mentioned processing in the subtitle display unit can be performed, for example, using AI, or can be performed without using AI. For example, the subtitle display unit can display the converted text on the lens of the glasses, using AR technology, to allow information to be visually grasped.

[0079] The voice recognition unit can estimate the user's emotion and adjust the accuracy of the voice recognition based on the emotion. The voice recognition unit can estimate the user's emotion using, for example, facial expression analysis and adjust the accuracy of the voice recognition based on the estimated user's emotion. The voice recognition unit can also estimate the user's emotion using, for example, voice analysis and adjust the accuracy of the voice recognition based on the estimated user's emotion. The voice recognition unit can also estimate the user's emotion using, for example, machine learning technology and adjust the accuracy of the voice recognition based on the estimated user's emotion. This allows for more accurate voice recognition by adjusting the accuracy of the voice recognition according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the voice recognition unit can estimate the user's emotion using facial expression analysis and adjust the accuracy of the voice recognition based on the estimated user's emotion.

[0080] The voice recognition unit can automatically filter background noise to improve the recognition accuracy. The voice recognition unit can automatically filter background noise using, for example, a noise canceling technology to improve the recognition accuracy. The voice recognition unit can also automatically filter background noise using, for example, a signal processing technology to improve the recognition accuracy. The voice recognition unit can also filter noise in a specific frequency band to improve the recognition accuracy. As a result, the accuracy of voice recognition is improved by filtering the background noise. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can automatically filter background noise using a noise canceling technology to improve the recognition accuracy.

[0081] The voice recognition unit can dynamically adjust the recognition algorithm according to the speaking speed of the speaker. The voice recognition unit adjusts the recognition algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. The voice recognition unit can also adjust the recognition algorithm according to the speaking speed of the speaker using, for example, adaptive filtering. The voice recognition unit can also adjust the recognition algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. As a result, the recognition accuracy is improved by adjusting the recognition algorithm according to the speaking speed of the speaker. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can adjust the recognition algorithm according to the speaking speed of the speaker using a dynamic adjustment algorithm to improve the recognition accuracy.

[0082] The voice recognition unit can estimate the user's emotion and determine the priority of voice recognition based on the emotion. The voice recognition unit can estimate the user's emotion using, for example, facial expression analysis and determine the priority of voice recognition based on the estimated user's emotion. The voice recognition unit can also estimate the user's emotion using, for example, voice analysis and determine the priority of voice recognition based on the estimated user's emotion. The voice recognition unit can also estimate the user's emotion using, for example, machine learning technology and determine the priority of voice recognition based on the estimated user's emotion. As a result, important voice can be preferentially recognized by determining the priority of voice recognition according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the voice recognition unit can estimate the user's emotion using facial expression analysis and determine the priority of voice recognition based on the estimated user's emotion.

[0083] The voice recognition unit can improve the recognition accuracy by taking into account the geographical accent of the speaker. The voice recognition unit can improve the recognition accuracy by taking into account the geographical accent of the speaker using, for example, a dialect model. The voice recognition unit can also improve the recognition accuracy by taking into account the geographical accent of the speaker using, for example, region-specific voice recognition. The voice recognition unit can also improve the recognition accuracy by taking into account the geographical accent of the speaker using, for example, a dialect model. As a result, the recognition accuracy is improved by taking into account the geographical accent of the speaker. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can improve the recognition accuracy by taking into account the geographical accent of the speaker using a dialect model.

[0084] The voice recognition unit can improve the recognition accuracy by referring to the speaker's past speech history. The voice recognition unit can improve the recognition accuracy by referring to the speaker's past speech history using, for example, a history database. The voice recognition unit can also improve the recognition accuracy by referring to the speaker's past speech history using, for example, a history analysis algorithm. The voice recognition unit can also improve the recognition accuracy by referring to the speaker's past speech history using, for example, a history database. As a result, the recognition accuracy is improved by referring to the speaker's past speech history. A part or all of the above-mentioned processing in the voice recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice recognition unit can improve the recognition accuracy by referring to the speaker's past speech history using a history database.

[0085] The conversion unit can estimate the user's emotion and adjust the expression method of the text conversion based on the emotion. The conversion unit can estimate the user's emotion using, for example, facial expression analysis, and adjust the expression method of the text conversion based on the estimated user's emotion. The conversion unit can also estimate the user's emotion using, for example, voice analysis, and adjust the expression method of the text conversion based on the estimated user's emotion. The conversion unit can also estimate the user's emotion using, for example, machine learning technology, and adjust the expression method of the text conversion based on the estimated user's emotion. As a result, more appropriate text is generated by adjusting the expression method of the text conversion according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the conversion unit can estimate the user's emotion using facial expression analysis, and adjust the expression method of the text conversion based on the estimated user's emotion.

[0086] The conversion unit can analyze the context of the voice and convert it into more natural text. The conversion unit can analyze the context of the voice using, for example, natural language processing technology and convert it into more natural text. The conversion unit can also analyze the context of the voice using, for example, a context model and convert it into more natural text. The conversion unit can also analyze the context of the voice using, for example, natural language processing technology and convert it into more natural text. As a result, more natural text is generated by analyzing the context of the voice. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can analyze the context of the voice using natural language processing technology and convert it into more natural text.

[0087] The conversion unit can automatically recognize technical terms and slang and convert them into appropriate text. The conversion unit can automatically recognize technical terms using, for example, a technical dictionary and convert them into appropriate text. The conversion unit can also automatically recognize slang using, for example, a slang database and convert them into appropriate text. The conversion unit can automatically recognize technical terms using, for example, a technical dictionary and convert them into appropriate text. As a result, more understandable text is generated by appropriately converting technical terms and slang. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can automatically recognize technical terms using a technical dictionary and convert them into appropriate text.

[0088] The conversion unit can estimate the user's emotion and adjust the length of the text conversion based on the emotion. The conversion unit can estimate the user's emotion using, for example, facial expression analysis, and adjust the length of the text conversion based on the estimated user's emotion. The conversion unit can also estimate the user's emotion using, for example, voice analysis, and adjust the length of the text conversion based on the estimated user's emotion. The conversion unit can also estimate the user's emotion using, for example, machine learning technology, and adjust the length of the text conversion based on the estimated user's emotion. As a result, more appropriate text is generated by adjusting the length of the text conversion according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the conversion unit can estimate the user's emotion using facial expression analysis, and adjust the length of the text conversion based on the estimated user's emotion.

[0089] The conversion unit can analyze the intention of the speaker of the voice and convert it into appropriate text. The conversion unit can analyze the intention of the speaker of the voice using, for example, an intention recognition algorithm and convert it into appropriate text. The conversion unit can also analyze the intention of the speaker of the voice using, for example, a natural language processing technology and convert it into appropriate text. The conversion unit can also analyze the intention of the speaker of the voice using, for example, an intention recognition algorithm and convert it into appropriate text. In this way, more appropriate text is generated by analyzing the intention of the speaker. A part or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can analyze the intention of the speaker of the voice using an intention recognition algorithm and convert it into appropriate text.

[0090] The conversion unit can convert the speech into text while taking into account background information of the speech. The conversion unit can convert the speech into text while taking into account background information of the speech, for example, using context analysis. The conversion unit can also convert the speech into text while taking into account background information of the speech, for example, using extraction of related information. The conversion unit can also convert the speech into text while taking into account background information of the speech, for example, using context analysis. As a result, more appropriate text is generated by taking into account the background information of the speech. A part or all of the above-mentioned processing in the conversion unit can be performed, for example, using AI, or can be performed without using AI. For example, the conversion unit can convert the speech into text while taking into account background information of the speech, using context analysis.

[0091] The subtitle display unit can estimate the user's emotion and adjust the display method of the subtitles based on the emotion. The subtitle display unit can estimate the user's emotion using, for example, facial expression analysis and adjust the display method of the subtitles based on the estimated user's emotion. The subtitle display unit can also estimate the user's emotion using, for example, voice analysis and adjust the display method of the subtitles based on the estimated user's emotion. The subtitle display unit can also estimate the user's emotion using, for example, machine learning technology and adjust the display method of the subtitles based on the estimated user's emotion. As a result, more appropriate subtitles are displayed by adjusting the display method of the subtitles according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the subtitle display unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the subtitle display unit can estimate the user's emotion using facial expression analysis and adjust the display method of the subtitles based on the estimated user's emotion.

[0092] The subtitle display unit can track the user's gaze and determine an appropriate display position. The subtitle display unit can track the user's gaze using, for example, an eye-tracking sensor, and determine an appropriate display position. The subtitle display unit can also track the user's gaze using, for example, an eye-tracking analysis algorithm, and determine an appropriate display position. The subtitle display unit can track the user's gaze using, for example, an eye-tracking sensor, and determine an appropriate display position. As a result, by tracking the user's gaze, subtitles with high visibility are displayed. A part or all of the above-mentioned processing in the subtitle display unit may be performed using, for example, AI, or may be performed without using AI. For example, the subtitle display unit can track the user's gaze using an eye-tracking sensor, and determine an appropriate display position.

[0093] The subtitle display unit can dynamically adjust the font and size of the text to be displayed. The subtitle display unit can adjust the font and size of the text to be displayed, for example, by using a dynamic layout engine. The subtitle display unit can also adjust the font and size of the text to be displayed, for example, by using adaptive font size adjustment. The subtitle display unit can also adjust the font and size of the text to be displayed, for example, by using a dynamic layout engine. This improves visibility by dynamically adjusting the font and size. A part or all of the above-mentioned processing in the subtitle display unit can be performed, for example, by using AI or without using AI. For example, the subtitle display unit can adjust the font and size of the text to be displayed, by using a dynamic layout engine.

[0094] The subtitle display unit can estimate the user's emotion and adjust the display timing of the subtitles based on the emotion. The subtitle display unit can estimate the user's emotion using, for example, facial expression analysis and adjust the display timing of the subtitles based on the estimated user's emotion. The subtitle display unit can also estimate the user's emotion using, for example, voice analysis and adjust the display timing of the subtitles based on the estimated user's emotion. The subtitle display unit can also estimate the user's emotion using, for example, machine learning technology and adjust the display timing of the subtitles based on the estimated user's emotion. As a result, the subtitles are displayed at a more appropriate timing by adjusting the display timing of the subtitles according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the subtitle display unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the subtitle display unit can estimate the user's emotion using facial expression analysis and adjust the display timing of the subtitles based on the estimated user's emotion.

[0095] The subtitle display unit can adopt a display method for reducing visual fatigue of the user. The subtitle display unit can reduce visual fatigue, for example, by using blue light cut. The subtitle display unit can also reduce visual fatigue, for example, by using adaptive brightness adjustment. The subtitle display unit can also reduce visual fatigue, for example, by using blue light cut. This reduces visual fatigue, making it possible to use the device for a long time. A part or all of the above-mentioned processing in the subtitle display unit can be performed, for example, by using AI, or can be performed without using AI. For example, the subtitle display unit can reduce visual fatigue by using blue light cut.

[0096] The subtitle display unit can adjust the brightness of the display in consideration of the user's ambient light. The subtitle display unit can adjust the brightness of the display in consideration of the user's ambient light, for example, using a light sensor. The subtitle display unit can also adjust the brightness of the display in consideration of the user's ambient light, for example, using adaptive brightness adjustment. The subtitle display unit can also adjust the brightness of the display in consideration of the user's ambient light, for example, using a light sensor. Thereby, the visibility is improved by adjusting the brightness of the display according to the ambient light. A part or all of the above-mentioned processing in the subtitle display unit may be performed, for example, using AI, or may be performed without using AI. For example, the subtitle display unit can adjust the brightness of the display in consideration of the user's ambient light, using a light sensor.

[0097] The face authentication unit can estimate the user's emotion and adjust the accuracy of face authentication based on the emotion. The face authentication unit can estimate the user's emotion using, for example, facial expression analysis and adjust the accuracy of face authentication based on the estimated user's emotion. The face authentication unit can also estimate the user's emotion using, for example, voice analysis and adjust the accuracy of face authentication based on the estimated user's emotion. The face authentication unit can also estimate the user's emotion using, for example, machine learning technology and adjust the accuracy of face authentication based on the estimated user's emotion. This allows for more accurate authentication by adjusting the accuracy of face authentication according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the face authentication unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the face authentication unit can estimate the user's emotion using facial expression analysis and adjust the accuracy of face authentication based on the estimated user's emotion.

[0098] The face authentication unit can improve the authentication accuracy by taking into account changes in facial expression. The face authentication unit can improve the authentication accuracy by taking into account changes in facial expression using, for example, a facial expression recognition algorithm. The face authentication unit can also improve the authentication accuracy by taking into account changes in facial expression using, for example, dynamic facial expression analysis. The face authentication unit can also improve the authentication accuracy by taking into account changes in facial expression using, for example, a facial expression recognition algorithm. As a result, the authentication accuracy is improved by taking into account changes in facial expression. A part or all of the above-mentioned processing in the face authentication unit may be performed using, for example, AI, or may be performed without using AI. For example, the face authentication unit can improve the authentication accuracy by taking into account changes in facial expression using a facial expression recognition algorithm.

[0099] The face authentication unit can recognize and identify multiple faces simultaneously. The face authentication unit can recognize and identify multiple faces simultaneously, for example, using a multi-face recognition algorithm. The face authentication unit can also recognize and identify multiple faces simultaneously, for example, using a parallel processing technique. The face authentication unit can also recognize and identify multiple faces simultaneously, for example, using a multi-face recognition algorithm. This improves authentication efficiency by recognizing multiple faces simultaneously. A part or all of the above-mentioned processing in the face authentication unit may be performed, for example, using AI, or may be performed without using AI. For example, the face authentication unit can recognize and identify multiple faces simultaneously, using a multi-face recognition algorithm.

[0100] The face authentication unit can estimate the user's emotion and determine the priority of face authentication based on the emotion. The face authentication unit can estimate the user's emotion using, for example, facial expression analysis and determine the priority of face authentication based on the estimated user's emotion. The face authentication unit can also estimate the user's emotion using, for example, voice analysis and determine the priority of face authentication based on the estimated user's emotion. The face authentication unit can also estimate the user's emotion using, for example, machine learning technology and determine the priority of face authentication based on the estimated user's emotion. Thereby, by determining the priority of face authentication according to the user's emotion, important faces can be preferentially authenticated. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the face authentication unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the face authentication unit can estimate the user's emotion using facial expression analysis and determine the priority of face authentication based on the estimated user's emotion.

[0101] The face authentication unit can improve the authentication accuracy by taking into account the angle of the face and the lighting conditions. The face authentication unit can improve the authentication accuracy by taking into account the angle of the face using, for example, 3D face recognition. The face authentication unit can also improve the authentication accuracy by taking into account the lighting conditions using, for example, a lighting correction algorithm. The face authentication unit can also improve the authentication accuracy by taking into account the angle of the face using, for example, 3D face recognition. As a result, the authentication accuracy is improved by taking into account the angle of the face and the lighting conditions. A part or all of the above-mentioned processing in the face authentication unit may be performed using, for example, AI, or may be performed without using AI. For example, the face authentication unit can improve the authentication accuracy by taking into account the angle of the face using 3D face recognition.

[0102] The face authentication unit can improve the authentication accuracy by referring to the past authentication history. The face authentication unit can improve the authentication accuracy by referring to the past authentication history, for example, using a history database. The face authentication unit can also improve the authentication accuracy by referring to the past authentication history, for example, using a history analysis algorithm. The face authentication unit can also improve the authentication accuracy by referring to the past authentication history, for example, using a history database. As a result, the authentication accuracy is improved by referring to the past authentication history. A part or all of the above-mentioned processing in the face authentication unit may be performed, for example, using AI, or may be performed without using AI. For example, the face authentication unit can improve the authentication accuracy by referring to the past authentication history, using a history database.

[0103] The lip reading unit can estimate the user's emotion and adjust the accuracy of lip reading based on the emotion. The lip reading unit can estimate the user's emotion using, for example, facial expression analysis and adjust the accuracy of lip reading based on the estimated user's emotion. The lip reading unit can also estimate the user's emotion using, for example, voice analysis and adjust the accuracy of lip reading based on the estimated user's emotion. The lip reading unit can also estimate the user's emotion using, for example, machine learning technology and adjust the accuracy of lip reading based on the estimated user's emotion. This allows for more accurate lip reading by adjusting the accuracy of lip reading according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the lip reading unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the lip reading unit may use facial expression analysis to infer a user's emotion and adjust the accuracy of the lip reading based on the inferred user's emotion.

[0104] The lip reading unit can analyze the movement of the speaker's lips at high resolution to improve accuracy. The lip reading unit can analyze the movement of the speaker's lips at high resolution, for example, using a high resolution camera. The lip reading unit can also analyze the movement of the speaker's lips at high resolution, for example, using an image processing algorithm. The lip reading unit can also analyze the movement of the speaker's lips at high resolution, for example, using a high resolution camera. This improves the accuracy of lip reading by analyzing the movement of the lips at high resolution. A part or all of the above-mentioned processing in the lip reading unit can be performed, for example, using AI, or can be performed without using AI. For example, the lip reading unit can analyze the movement of the speaker's lips at high resolution using a high resolution camera to improve the accuracy of lip reading.

[0105] The lip reading unit can dynamically adjust the analysis algorithm according to the speaking speed of the speaker. The lip reading unit can adjust the analysis algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. The lip reading unit can also adjust the analysis algorithm according to the speaking speed of the speaker using, for example, adaptive filtering. The lip reading unit can also adjust the analysis algorithm according to the speaking speed of the speaker using, for example, a dynamic adjustment algorithm. Thereby, by adjusting the analysis algorithm according to the speaking speed, the accuracy of lip reading is improved. A part or all of the above-mentioned processing in the lip reading unit can be performed using, for example, AI, or can be performed without using AI. For example, the lip reading unit can adjust the analysis algorithm according to the speaking speed of the speaker using a dynamic adjustment algorithm to improve the accuracy of lip reading.

[0106] The lip reading unit can estimate the user's emotion and determine the priority of lip reading based on the emotion. The lip reading unit can estimate the user's emotion using, for example, facial expression analysis and determine the priority of lip reading based on the estimated user's emotion. The lip reading unit can also estimate the user's emotion using, for example, voice analysis and determine the priority of lip reading based on the estimated user's emotion. The lip reading unit can also estimate the user's emotion using, for example, machine learning technology and determine the priority of lip reading based on the estimated user's emotion. Thereby, by determining the priority of lip reading according to the user's emotion, important lip movements can be preferentially analyzed. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the lip reading unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the lip reading unit may use facial expression analysis to infer a user's emotion and determine lip reading priorities based on the inferred user's emotion.

[0107] The lip reading unit can improve the analysis accuracy by taking into account the change in the speaker's facial expression. The lip reading unit can improve the analysis accuracy by taking into account the change in the speaker's facial expression using, for example, a facial expression recognition algorithm. The lip reading unit can also improve the analysis accuracy by taking into account the change in the speaker's facial expression using, for example, a dynamic facial expression analysis. The lip reading unit can also improve the analysis accuracy by taking into account the change in the speaker's facial expression using, for example, a facial expression recognition algorithm. Thereby, the accuracy of lip reading is improved by taking into account the change in the facial expression. A part or all of the above-mentioned processing in the lip reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the lip reading unit can improve the analysis accuracy by taking into account the change in the speaker's facial expression using a facial expression recognition algorithm.

[0108] The lip reading unit can improve the analysis accuracy by referring to the speaker's past speech history. The lip reading unit can improve the analysis accuracy by referring to the speaker's past speech history using, for example, a history database. The lip reading unit can also improve the analysis accuracy by referring to the speaker's past speech history using, for example, a history analysis algorithm. The lip reading unit can also improve the analysis accuracy by referring to the speaker's past speech history using, for example, a history database. As a result, the accuracy of lip reading is improved by referring to the past speech history. A part or all of the above-mentioned processing in the lip reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the lip reading unit can improve the analysis accuracy by referring to the speaker's past speech history using a history database.

[0109] The translation unit can estimate the user's emotion and adjust the translation expression method based on the emotion. The translation unit can estimate the user's emotion using, for example, facial expression analysis, and adjust the translation expression method based on the estimated user's emotion. The translation unit can also estimate the user's emotion using, for example, voice analysis, and adjust the translation expression method based on the estimated user's emotion. The translation unit can also estimate the user's emotion using, for example, machine learning technology, and adjust the translation expression method based on the estimated user's emotion. As a result, a more appropriate translation is provided by adjusting the translation expression method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the translation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the translation unit can estimate the user's emotion using facial expression analysis, and adjust the translation expression method based on the estimated user's emotion.

[0110] The translation unit can analyze the context to provide a more natural translation. The translation unit can analyze the context using, for example, natural language processing technology to provide a more natural translation. The translation unit can also analyze the context using, for example, a context model to provide a more natural translation. The translation unit can also analyze the context using, for example, natural language processing technology to provide a more natural translation. Thereby, a more natural translation is provided by analyzing the context. A part or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without using AI. For example, the translation unit can analyze the context using natural language processing technology to provide a more natural translation.

[0111] The translation unit can automatically recognize technical terms and slang and provide an appropriate translation. The translation unit can automatically recognize technical terms using, for example, a technical dictionary and provide an appropriate translation. The translation unit can also automatically recognize slang using, for example, a slang database and provide an appropriate translation. The translation unit can automatically recognize technical terms using, for example, a technical dictionary and provide an appropriate translation. As a result, a translation that is easier to understand is provided by appropriately translating technical terms and slang. Some or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without using AI. For example, the translation unit can automatically recognize technical terms using a technical dictionary and provide an appropriate translation.

[0112] The translation unit can estimate the user's emotion and determine the translation priority based on the emotion. The translation unit can estimate the user's emotion using, for example, facial expression analysis and determine the translation priority based on the estimated user's emotion. The translation unit can also estimate the user's emotion using, for example, voice analysis and determine the translation priority based on the estimated user's emotion. The translation unit can also estimate the user's emotion using, for example, machine learning technology and determine the translation priority based on the estimated user's emotion. As a result, important information can be preferentially translated by determining the translation priority according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the translation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the translation unit can estimate the user's emotion using facial expression analysis and determine the translation priority based on the estimated user's emotion.

[0113] The translation unit can analyze the intention of the speaker and provide an appropriate translation. The translation unit can analyze the intention of the speaker using, for example, an intention recognition algorithm, and provide an appropriate translation. The translation unit can also analyze the intention of the speaker using, for example, a natural language processing technique, and provide an appropriate translation. The translation unit can also analyze the intention of the speaker using, for example, an intention recognition algorithm, and provide an appropriate translation. In this way, a more appropriate translation is provided by analyzing the intention of the speaker. A part or all of the above-mentioned processing in the translation unit may be performed using, for example, AI, or may be performed without using AI. For example, the translation unit can analyze the intention of the speaker using an intention recognition algorithm, and provide an appropriate translation.

[0114] The translation unit can provide a translation taking into account background information of the speaker. The translation unit can provide a translation taking into account background information of the speaker, for example, using context analysis. The translation unit can also provide a translation taking into account background information of the speaker, for example, using related information extraction. The translation unit can also provide a translation taking into account background information of the speaker, for example, using context analysis. Thereby, by taking into account background information of the speaker, a more appropriate translation is provided. Some or all of the above-mentioned processing in the translation unit can be performed, for example, using AI, or can be performed without using AI. For example, the translation unit can provide a translation taking into account background information of the speaker, using context analysis.

[0115] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0116] The voice recognition unit can estimate the user's emotion and adjust the accuracy of voice recognition based on the emotion. For example, the voice recognition unit can estimate the user's emotion using facial expression analysis and adjust the accuracy of voice recognition based on the estimated user's emotion. The voice recognition unit can also estimate the user's emotion using voice analysis and adjust the accuracy of voice recognition based on the estimated user's emotion. The voice recognition unit can also estimate the user's emotion using machine learning technology and adjust the accuracy of voice recognition based on the estimated user's emotion. This allows for more accurate voice recognition by adjusting the accuracy of voice recognition according to the user's emotion.

[0117] The speech recognition unit can improve the recognition accuracy by taking into account the geographic accent of the speaker. For example, the speech recognition unit improves the recognition accuracy by taking into account the geographic accent of the speaker using a dialect model. The speech recognition unit can also improve the recognition accuracy by taking into account the geographic accent of the speaker using region-specific speech recognition. The speech recognition unit can also improve the recognition accuracy by taking into account the geographic accent of the speaker using a dialect model. As a result, the recognition accuracy is improved by taking into account the geographic accent of the speaker.

[0118] The conversion unit can estimate the user's emotion and adjust the expression method of the text conversion based on the emotion. For example, the conversion unit can estimate the user's emotion using facial expression analysis and adjust the expression method of the text conversion based on the estimated user's emotion. The conversion unit can also estimate the user's emotion using voice analysis and adjust the expression method of the text conversion based on the estimated user's emotion. The conversion unit can also estimate the user's emotion using machine learning technology and adjust the expression method of the text conversion based on the estimated user's emotion. In this way, by adjusting the expression method of the text conversion according to the user's emotion, more appropriate text is generated.

[0119] The subtitle display unit can track the user's gaze and determine an appropriate display position. For example, the subtitle display unit uses an eye-tracking sensor to track the user's gaze and determine an appropriate display position. The subtitle display unit can also use an eye-tracking analysis algorithm to track the user's gaze and determine an appropriate display position. The subtitle display unit can also use an eye-tracking sensor to track the user's gaze and determine an appropriate display position. In this way, highly visible subtitles are displayed by tracking the user's gaze.

[0120] The subtitle display unit can estimate the user's emotion and adjust the subtitle display method based on the emotion. For example, the subtitle display unit can estimate the user's emotion using facial expression analysis and adjust the subtitle display method based on the estimated user's emotion. The subtitle display unit can also estimate the user's emotion using voice analysis and adjust the subtitle display method based on the estimated user's emotion. The subtitle display unit can also estimate the user's emotion using machine learning technology and adjust the subtitle display method based on the estimated user's emotion. In this way, more appropriate subtitles are displayed by adjusting the subtitle display method according to the user's emotion.

[0121] The face authentication unit can recognize and identify multiple faces simultaneously. For example, the face authentication unit uses a multi-face recognition algorithm to recognize and identify multiple faces simultaneously. The face authentication unit can also use a parallel processing technique to recognize and identify multiple faces simultaneously. The face authentication unit can also use a multi-face recognition algorithm to recognize and identify multiple faces simultaneously. This improves authentication efficiency by recognizing multiple faces simultaneously.

[0122] The face authentication unit can estimate the user's emotion and determine the priority of face authentication based on the emotion. For example, the face authentication unit can estimate the user's emotion using facial expression analysis and determine the priority of face authentication based on the estimated user's emotion. The face authentication unit can also estimate the user's emotion using voice analysis and determine the priority of face authentication based on the estimated user's emotion. The face authentication unit can also estimate the user's emotion using machine learning technology and determine the priority of face authentication based on the estimated user's emotion. In this way, important faces can be preferentially authenticated by determining the priority of face authentication according to the user's emotion.

[0123] The lip reading unit may analyze the speaker's lip movements at high resolution to improve accuracy. For example, the lip reading unit may use a high resolution camera to analyze the speaker's lip movements at high resolution. The lip reading unit may also use an image processing algorithm to analyze the speaker's lip movements at high resolution. The lip reading unit may also use a high resolution camera to analyze the speaker's lip movements at high resolution. Thus, by analyzing the lip movements at high resolution, the accuracy of lip reading is improved.

[0124] The lip reading unit can estimate the user's emotion and determine the priority of lip reading based on the emotion. For example, the lip reading unit can estimate the user's emotion using facial expression analysis and determine the priority of lip reading based on the estimated user's emotion. The lip reading unit can also estimate the user's emotion using voice analysis and determine the priority of lip reading based on the estimated user's emotion. The lip reading unit can also estimate the user's emotion using machine learning technology and determine the priority of lip reading based on the estimated user's emotion. In this way, important lip movements can be preferentially analyzed by determining the priority of lip reading according to the user's emotion.

[0125] The translation unit can estimate the user's emotion and adjust the translation expression method based on the emotion. For example, the translation unit can estimate the user's emotion using facial expression analysis and adjust the translation expression method based on the estimated user's emotion. The translation unit can also estimate the user's emotion using voice analysis and adjust the translation expression method based on the estimated user's emotion. The translation unit can also estimate the user's emotion using machine learning technology and adjust the translation expression method based on the estimated user's emotion. In this way, a more appropriate translation can be provided by adjusting the translation expression method according to the user's emotion.

[0126] The process flow of the second embodiment will be briefly described below.

[0127] Step 1: The speech recognition unit recognizes speech. The speech recognition unit can recognize speech using, for example, an acoustic model, a language model, deep learning technology, a speech recognition engine, or natural language processing technology. Step 2: The conversion unit converts the voice recognized by the voice recognition unit into text. The conversion unit can convert the voice into text using, for example, a voice recognition engine or natural language processing technology. Step 3: The subtitle display unit displays the text converted by the conversion unit on the wearable device. The subtitle display unit displays the text on a wearable device such as smart glasses or a smart watch. In addition, the text can be displayed using AR technology or display technology.

[0128] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0129] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0130] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0131] For example, the voice recognition unit is realized by the processor 46 of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the conversion unit is realized by the processor 46 of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the subtitle display unit is realized by the display 40A of the smart device 14. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0132] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0133] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0134] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0135] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0136] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0137] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0138] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0139] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0140] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0141] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0142] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0143] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0144] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0145] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0146] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0147] For example, the voice recognition unit is realized by the processor 46 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the conversion unit is realized by the processor 46 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the subtitle display unit is realized by the display of the smart glasses 214. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0148] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0149] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0150] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0151] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0152] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0153] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0154] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0155] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0156] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0157] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0158] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0159] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0160] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0161] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0162] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0163] For example, the voice recognition unit is realized by the processor 46 of the headset type terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the conversion unit is realized by the processor 46 of the headset type terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the subtitle display unit is realized by the display 343 of the headset type terminal 314. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0164] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0165] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0166] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0167] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0168] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0169] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0170] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0171] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0172] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0173] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0174] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0175] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0176] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0177] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0178] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0179] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0180] For example, the voice recognition unit is realized by the processor 46 of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the conversion unit is realized by the processor 46 of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the subtitle display unit is realized by the display of the robot 414. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0181] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0182] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0183] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0184] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0185] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0186] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0187] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0188] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0189] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0190] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0191] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0192] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0193] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0194] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0195] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0196] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0197] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0198] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0199] (Appendix 1) A voice recognition unit that recognizes voice; a conversion unit that converts the voice recognized by the voice recognition unit into text; a subtitle display unit that displays the text converted by the conversion unit on a wearable device. A system characterized by: (Appendix 2) A face authentication unit that authenticates a face of a person who speaks the voice; The apparatus further includes a unit for performing lip reading according to a person identified by the face authentication unit. 2. The system of claim 1, (Appendix 3) The conversion unit is Converting environmental sound information into text 2. The system of claim 1, (Appendix 4) Equipped with a translation function 2. The system of claim 1, (Appendix 5) The voice recognition unit Instant speech recognition 2. The system of claim 1, (Appendix 6) The subtitle display unit includes: Display the converted text on the glasses lens 2. The system of claim 1, (Appendix 7) The voice recognition unit Estimate the user's emotions and adjust the accuracy of speech recognition based on those emotions 2. The system of claim 1, (Appendix 8) The voice recognition unit Automatically filter background noise to improve recognition accuracy 2. The system of claim 1, (Appendix 9) The voice recognition unit Dynamically adjusts recognition algorithms based on the speaker's speaking rate 2. The system of claim 1, (Appendix 10) The voice recognition unit Estimate user emotions and prioritize speech recognition based on those emotions 2. The system of claim 1, (Appendix 11) The voice recognition unit Improve recognition accuracy by considering the speaker's geographic accent 2. The system of claim 1, (Appendix 12) The voice recognition unit Improve recognition accuracy by referencing the speaker's past speech history 2. The system of claim 1, (Appendix 13) The conversion unit is Infer user sentiment and adjust the text conversion based on that sentiment 2. The system of claim 1, (Appendix 14) The conversion unit is Analyze the context of speech and convert it into more natural-sounding text 2. The system of claim 1, (Appendix 15) The conversion unit is Automatically recognizes technical terms and slang and converts them into appropriate text 2. The system of claim 1, (Appendix 16) The conversion unit is Infer user sentiment and adjust the length of the text conversion based on that sentiment 2. The system of claim 1, (Appendix 17) The conversion unit is Analyze the speaker's intent and convert it into appropriate text 2. The system of claim 1, (Appendix 18) The conversion unit is Convert audio into text taking into account background information 2. The system of claim 1, (Appendix 19) The subtitle display unit includes: Infer user emotions and adjust the way subtitles are displayed based on those emotions 2. The system of claim 1, (Appendix 20) The subtitle display unit includes: Tracking the user's gaze to determine the appropriate display position 2. The system of claim 1, (Appendix 21) The subtitle display unit includes: Dynamically adjust the font and size of the text you display 2. The system of claim 1, (Appendix 22) The subtitle display unit includes: Estimate user emotions and adjust subtitle display timing based on those emotions 2. The system of claim 1, (Appendix 23) The subtitle display unit includes: Adopt a display method to reduce the user's visual fatigue 2. The system of claim 1, (Appendix 24) The subtitle display unit includes: Adjust display brightness to take into account the user's ambient light 2. The system of claim 1, (Appendix 25) The face authentication unit is Estimate user emotions and adjust facial recognition accuracy based on those emotions 3. The system of claim 2, (Appendix 26) The face authentication unit is Improve authentication accuracy by taking into account changes in facial expressions 3. The system of claim 2, (Appendix 27) The face authentication unit is Recognize and identify multiple faces simultaneously 3. The system of claim 2, (Appendix 28) The face authentication unit is Estimate user emotions and prioritize face recognition based on those emotions 3. The system of claim 2, (Appendix 29) The face authentication unit is Improve authentication accuracy by taking into account face angle and lighting conditions 3. The system of claim 2, (Appendix 30) The face authentication unit is Improve authentication accuracy by referencing past authentication history 3. The system of claim 2, (Appendix 31) The lip reading unit includes: Infer the user's emotions and adjust lip-reading accuracy based on those emotions 3. The system of claim 2, (Appendix 32) The lip reading unit includes: Improve accuracy by analyzing the speaker's lip movements at high resolution 3. The system of claim 2, (Appendix 33) The lip reading unit includes: Dynamically adjust analysis algorithms according to the speaker's speaking rate 3. The system of claim 2, (Appendix 34) The lip reading unit includes: Infer user emotions and prioritize lip reading based on those emotions 3. The system of claim 2, (Appendix 35) The lip reading unit includes: Improve analysis accuracy by taking into account changes in the speaker's facial expressions 3. The system of claim 2, (Appendix 36) The lip reading unit includes: Improve analysis accuracy by referencing the speaker's past speech history 3. The system of claim 2, (Appendix 37) The translation unit is Infer user emotions and adjust translations based on those emotions 4. The system of claim 3. (Appendix 38) The translation unit is Analyzes context to provide more natural translations 4. The system of claim 3. (Appendix 39) The translation unit is Automatically recognizes technical terms and slang and provides appropriate translations 4. The system of claim 3. (Appendix 40) The translation unit is Estimate user sentiment and prioritize translations based on that sentiment 4. The system of claim 3. (Appendix 41) The translation unit is Analyzes speaker intent and provides appropriate translation 4. The system of claim 3. (Appendix 42) The translation unit is Provide translations that take into account the speaker's context 4. The system of claim 3. [Explanation of symbols]

[0200] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A voice recognition unit that recognizes voice; a conversion unit that converts the voice recognized by the voice recognition unit into text; a subtitle display unit that displays the text converted by the conversion unit on a wearable device. A system characterized by:

2. A face authentication unit that authenticates a face of a person who speaks the voice; The apparatus further includes a unit for performing lip reading according to a person identified by the face authentication unit.

2. The system of claim 1 .

3. The conversion unit is Converting environmental sound information into text 2. The system of claim 1 .

4. Equipped with a translation function 2. The system of claim 1 .

5. The subtitle display unit includes: Display the converted text on the glasses lens 2. The system of claim 1 .

6. The voice recognition unit Estimate the user's emotions and adjust the accuracy of speech recognition based on those emotions 2. The system of claim 1 .

7. The voice recognition unit Automatically filter background noise to improve recognition accuracy 2. The system of claim 1 .

8. The voice recognition unit Dynamically adjusts recognition algorithms based on the speaker's speaking rate 2. The system of claim 1 .

9. The voice recognition unit Estimate user emotions and prioritize speech recognition based on those emotions 2. The system of claim 1 .

Images