system

Abstract

The system according to this embodiment aims to efficiently operate smartphones using voice input, transfer settings when changing models, and support multiple languages. [Solution] The system according to the embodiment comprises a reception unit, an analysis unit, a processing unit, a handover unit, and an understanding unit. The reception unit receives voice input. The analysis unit analyzes the voice input received by the reception unit. The processing unit performs processing based on the results analyzed by the analysis unit. The handover unit handles settings and application transfer when changing devices. The understanding unit handles multilingual support and dialect / accent understanding.

Description

Technical Field

[0006]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, operations of a smartphone by voice input, setting inheritance when changing the model, and multi-language support are not sufficiently performed, and there is room for improvement.

[0005] The system according to the embodiment aims to efficiently perform operations of a smartphone by voice input, setting inheritance when changing the model, and multi-language support.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a reception unit, an analysis unit, a processing unit, a handover unit, and an understanding unit. The reception unit receives voice input. The analysis unit analyzes the voice input received by the reception unit. The processing unit performs processing based on the results analyzed by the analysis unit. The handover unit handles settings and application transfer when changing devices. The understanding unit handles multilingual support and dialect / accent understanding. [Effects of the Invention]

[0007] The system according to this embodiment can efficiently operate a smartphone using voice input, transfer settings when changing models, and support multiple languages. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.<0000​​​​​​​​​​The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The voice assistant agent system according to an embodiment of the present invention is a system that allows users to perform tasks such as adjusting smartphone settings, searching for information, online shopping, and booking flights and hotels through voice input. This voice assistant agent system can store user information regardless of the smartphone model by linking it to a smartphone model-independent ID (for example, a user account for an application or online service). As a result, user information is carried over even when the user changes models. Furthermore, when switching to a new device, settings and apps from the old device can be transferred to the new device, saving the user the trouble of setting them up again. In addition, this voice assistant agent system supports multiple languages ​​and can understand regional dialects and accents, so it can also be used as an interpreter. For example, the user makes a voice input. For example, the user may make a voice input such as "I want to change my smartphone settings," "I want to search for XX," "I want to buy XX," or "I want to book a flight." This voice input is analyzed by a generating AI and processed appropriately. Next, the generating AI processes tasks such as changing smartphone settings, searching for information, online shopping, and booking flights and hotels based on the voice input. For example, in the case of changing smartphone settings, the generating AI investigates how to change the settings and changes the necessary settings. For research, the generating AI searches the internet for information and provides results. For e-commerce shopping, the generating AI searches for products and completes the purchase process. For flight and hotel reservations, the generating AI searches for reservation sites and completes the reservation process. Furthermore, when changing devices, the generating AI retrieves settings and app information from the old device and transfers it to the new device. This allows the user to use the same settings and apps on the new device. In addition, this voice assistant agent system supports multiple languages ​​and can understand regional dialects and accents. For example, it supports multiple languages ​​such as English, Japanese, and Chinese, and even if the user provides voice input in a different language, the generating AI can appropriately analyze and process it. Furthermore, because it can understand regional dialects and accents, it can also be used as an interpreter.Thus, the voice assistant agent system of the present invention maximizes user convenience by storing user information independently of the device model, handling settings and app transfers when changing devices, and enabling multilingual support and understanding of dialects and accents. As a result, the voice assistant agent system can maximize user convenience.

[0029] The voice assistant agent system according to this embodiment comprises a reception unit, an analysis unit, a processing unit, a handover unit, and an understanding unit. The reception unit receives voice input. The reception unit receives voice input, for example, when a user speaks into a smartphone. The reception unit can also receive voice input through a microphone. Furthermore, the reception unit can accept voice input in various formats, such as natural language or command format. For example, the reception unit receives voice input when a user says, "I want to change the settings on my smartphone." The reception unit can also receive voice input when a user says, "I want to look up XX." Furthermore, the reception unit can also receive voice input when a user says, "I want to buy XX." The analysis unit analyzes the voice input received by the reception unit. The analysis unit converts the voice input into text data, for example, using speech recognition technology. The analysis unit can also analyze the content of the voice input using an analysis algorithm. For example, the analysis unit converts the user's voice input into text data using speech recognition technology. The analysis unit can also analyze the content of the voice input using an analysis algorithm. Furthermore, the analysis unit can also analyze the content of voice input using a generation AI. For example, the analysis unit inputs a prompt to the generation AI, "Please analyze the content of this voice input," and the generation AI analyzes the content of the voice input. The processing unit performs processing based on the results analyzed by the analysis unit. For example, the processing unit can change the settings of the smartphone. The processing unit can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and perform the purchase procedure. For example, the processing unit can change the settings of the smartphone. The processing unit can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and perform the purchase procedure. The transfer unit performs the transfer of settings and applications when changing devices. For example, the transfer unit can acquire settings and application information from the old device and transfer them to the new device. Furthermore, the transfer unit can perform the transfer according to the setting items and application types. For example, the transfer unit can acquire settings and application information from the old device and transfer them to the new device.Furthermore, the transfer unit can perform data transfer depending on the settings and type of application. The understanding unit handles multilingual support and understands dialects and accents. The understanding unit supports multiple languages, such as English, Japanese, and Chinese. It can also understand regional dialects and accents. For example, the understanding unit supports multiple languages, such as English, Japanese, and Chinese. It can also understand regional dialects and accents. As a result, the voice assistant agent system can analyze and process voice input, enabling settings and app transfers when changing devices, multilingual support, and understanding of dialects and accents.

[0030] The reception unit accepts voice input. For example, it accepts voice input when the user speaks into their smartphone. It can also receive voice input via a microphone. Furthermore, the reception unit can accept various voice input formats, including natural language and command formats. For example, it can accept voice input when the user says, "I want to change my smartphone settings." It can also accept voice input when the user says, "I want to look up XX." Furthermore, it can accept voice input when the user says, "I want to buy XX." The reception unit uses a high-sensitivity microphone to accept voice input and incorporates noise-canceling technology to eliminate ambient noise. This allows users to accurately input voice even in noisy environments. In addition, the reception unit can identify the characteristics of the user's voice during voice input and perform personal authentication. This enables the voice assistant agent system to provide customized services for each user. For example, if a user says, "I want to check my calendar," the reception unit recognizes the user's voice and displays the user's calendar information. Furthermore, the reception desk can handle complex requests by using natural language processing technology to understand the user's intent during voice input. For example, if a user says, "Tell me the weather for tomorrow," the reception desk will understand the request and provide appropriate information. This allows the reception desk to respond quickly and accurately to a wide range of user requests.

[0031] The analysis unit analyzes the voice input received by the reception unit. The analysis unit converts the voice input into text data using, for example, speech recognition technology. The analysis unit can also analyze the content of the voice input using analysis algorithms. For example, the analysis unit converts the user's voice input into text data using speech recognition technology. Furthermore, the analysis unit can analyze the content of the voice input using a generative AI. For example, the analysis unit inputs the prompt "Please analyze the content of this voice input" to the generative AI, which then analyzes the content of the voice input. The analysis unit employs a deep learning model as its speech recognition technology, achieving highly accurate speech recognition. This allows it to accurately convert the user's utterance into text data. Furthermore, the analysis unit can analyze the context and intent of the voice input using natural language processing technology. For example, if a user says "Tell me the weather for tomorrow," the analysis unit understands the context and performs appropriate processing to provide weather information. The analysis unit can also analyze the content of the voice input more advancedly using a generative AI. Generative AI learns from vast amounts of data and can deeply understand user utterances. For example, if a user says, "Prepare the materials for the next meeting," the analysis unit can use the generative AI to understand the request and issue instructions for preparing the appropriate materials. This allows the analysis unit to respond quickly and accurately to a wide range of user requests.

[0032] The processing unit performs processing based on the results analyzed by the analysis unit. For example, the processing unit can change the settings of a smartphone. It can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and carry out the purchase process. For example, the processing unit can change the settings of a smartphone. It can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and carry out the purchase process. The processing unit performs specific actions based on the text data and analysis results received from the analysis unit. For example, if a user says, "Adjust the brightness of my smartphone," the processing unit adjusts the brightness of the smartphone according to that instruction. Also, if a user says, "Find a nearby restaurant," the processing unit searches for information on the internet and provides information on nearby restaurants. Furthermore, if a user says, "I want to buy XX," the processing unit searches for that product and carries out the purchase process. The processing unit can respond flexibly to user requests. For example, if a user says, "Prepare the materials for the next meeting," the processing unit searches for the necessary materials according to that request and prepares them. In addition, the processing unit can provide more personalized services based on the user's past requests and behavioral history. This allows the processing unit to respond quickly and accurately to a wide range of user requests.

[0033] The data transfer unit handles the transfer of settings and apps when changing devices. For example, the data transfer unit retrieves settings and app information from the old device and transfers it to the new device. The data transfer unit can also perform the transfer according to the settings and app types. For example, the data transfer unit retrieves settings and app information from the old device and transfers it to the new device. The data transfer unit automates data transfer from the old device to the new device so that users can smoothly transition to the new device. For example, if a user purchases a new smartphone, the data transfer unit automatically retrieves settings and app information from the old smartphone and transfers it to the new smartphone. This allows the user to use the new device without any hassle. Furthermore, the data transfer unit can select the optimal transfer method according to the settings and app types. For example, it can prioritize the transfer of important settings and app data, and avoid transferring unnecessary data. In addition, the data transfer unit uses encryption technology to protect data in order to ensure data security. This ensures that users' personal information and important data are transferred safely. The data transfer section allows for efficient and secure data transfer, enabling users to smoothly transition to a new device.

[0034] The understanding unit handles multiple languages, including English, Japanese, and Chinese. It can also understand regional dialects and accents. The understanding unit accurately understands and responds appropriately to any language the user speaks in. For example, if a user speaks in English, the understanding unit accurately understands and responds in English. Similarly, if a user speaks in Japanese, the understanding unit accurately understands and responds in Japanese. Furthermore, the understanding unit can understand regional dialects and accents. For example, if a user speaks in Kansai dialect, the understanding unit accurately understands and responds appropriately. The understanding unit combines speech recognition technology and natural language processing technology to achieve multilingual support and dialect / accent understanding. This allows users to speak in their native language or regional dialect, and the voice assistant agent system accurately understands and responds appropriately. The understanding unit can provide a user-friendly system for a wider range of users by supporting diverse languages ​​and dialects.

[0035] The processing unit can change the settings of the smartphone. For example, the processing unit can access the smartphone's settings menu and change the settings according to the user's instructions. For example, if the user instructs the processing unit to "turn on Wi-Fi," the processing unit will turn on the Wi-Fi setting. The processing unit can also adjust the screen brightness if the user instructs it to "adjust the screen brightness." Furthermore, the processing unit can change the notification sound if the user instructs it to "change the notification sound." This makes it possible to change the settings of the smartphone. Some or all of the above processing in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input the user's instructions into a generating AI, which can then analyze and execute the procedure for changing the settings.

[0036] The processing unit can search for information on the internet and provide the results. For example, if the user instructs the processing unit to "research about XX," it will use an internet search engine to search for information. For example, if the user instructs the processing unit to "find the latest news," it will search news sites and provide the latest news. The processing unit can also search recipe sites and provide recipe information if the user instructs it to "find a recipe for XX." Furthermore, if the user instructs it to "find the weather for XX," it can search weather forecast sites and provide weather information. This makes it possible to search for information on the internet and provide the results. Some or all of the above processing in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input the user's instructions into a generating AI, which can then analyze and execute the information retrieval procedure.

[0037] The processing unit can search for products and carry out purchase procedures. For example, if the user instructs the processing unit to "buy XX," it will search e-commerce sites and find the product. For example, if the user instructs the processing unit to "buy a new smartphone," it will search e-commerce sites and provide smartphone product information. The processing unit can also search e-commerce sites for books and provide book product information if the user instructs to "buy a book about XX." Furthermore, if the user instructs to "buy clothes about XX," it can search e-commerce sites for fashion and provide clothing product information. This makes it possible to search for products and carry out purchase procedures. Some or all of the above-described processes in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input user instructions into a generating AI, which can analyze and execute the product search procedure.

[0038] The processing unit can search for reservation websites and perform reservation procedures. For example, if the user instructs the processing unit to "book a flight," it will search for airline reservation websites and perform the reservation procedure. For example, if the user instructs the processing unit to "book a hotel," it will search for hotel reservation websites and perform the reservation procedure. The processing unit can also search for restaurant reservation websites and perform the reservation procedure if the user instructs the processing unit to "book a restaurant." Furthermore, if the user instructs the processing unit to "book a rental car," it will search for rental car reservation websites and perform the reservation procedure. This makes it possible to search for reservation websites and perform reservation procedures. Some or all of the above-described processes in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input user instructions into a generating AI, which can then analyze and execute the reservation procedure steps.

[0039] The transfer unit can acquire settings and application information from the old device and transfer them to the new device. For example, when a user switches to a new smartphone, the transfer unit acquires settings and application information from the old device and transfers them to the new device. For example, the transfer unit acquires Wi-Fi settings, application installation status, user custom settings, etc. from the old device and transfers them to the new device. The transfer unit can also transfer data from applications that the user was using. For example, the transfer unit transfers the application settings and data from the old device to the new device. This makes it possible to transfer settings and application information from the old device to the new device. Some or all of the above processing in the transfer unit may be performed using AI, for example, or without AI. For example, the transfer unit can input the settings and application information from the old device into a generating AI, which can then analyze and execute the transfer procedure.

[0040] The understanding unit supports multiple languages, including English, Japanese, and Chinese, and can understand regional dialects and accents. For example, if a user speaks in English, the understanding unit analyzes the English voice input and processes it appropriately. For example, if a user speaks in Japanese, the understanding unit analyzes the Japanese voice input and processes it appropriately. The understanding unit can also analyze the Chinese voice input if a user speaks in Chinese and process it appropriately. Furthermore, the understanding unit can understand regional dialects and accents. For example, if a user speaks in Kansai dialect, the understanding unit analyzes the Kansai dialect voice input and processes it appropriately. The understanding unit can also understand the accent if a user speaks with a Southern American accent and process it appropriately. This enables multilingual support and the understanding of regional dialects and accents. Some or all of the processing described above in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's voice input into a generating AI, which can handle multilingual support and understand dialects and accents.

[0041] The reception unit can analyze the user's past voice input history and select the optimal reception method. For example, the reception unit prioritizes receiving voice commands that the user has frequently used in the past. For example, the reception unit can predict commands to be used during specific time periods based on the user's past voice input history and adjust the reception method accordingly. The reception unit can also analyze the user's voice input patterns and select the optimal speech recognition algorithm. This makes it possible to analyze the user's past voice input history and select the optimal reception method. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit can input the user's past voice input history into a generating AI, which can then select the optimal reception method.

[0042] The reception unit can filter voice input based on the user's current situation and areas of interest. For example, the reception unit may accept only relevant voice commands based on the user's current situation. For example, the reception unit may prioritize accepting specific voice commands based on the user's areas of interest. The reception unit can also filter voice commands related to the user's current activity and accept only appropriate commands. This makes it possible to filter voice input based on the user's current situation and areas of interest. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit may input the user's current situation and areas of interest into a generating AI, which can then perform the filtering.

[0043] The reception unit can prioritize receiving voice input by considering the user's geographical location information. For example, if the user is in a specific location, the reception unit will prioritize receiving voice commands related to that location. For example, based on the user's current location, the reception unit will prioritize receiving voice commands related to nearby information. Furthermore, if the user is traveling, the reception unit can also prioritize receiving voice commands related to their travel destination. This makes it possible to prioritize receiving voice input that is highly relevant by considering the user's geographical location information. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit can input the user's geographical location information into a generating AI, which can then prioritize receiving input that is highly relevant.

[0044] The reception unit can analyze the user's social media activity and accept relevant input when receiving voice input. For example, the reception unit can prioritize receiving relevant voice commands based on the content of the user's social media posts. For example, the reception unit can prioritize receiving relevant voice commands based on the activity of the user's social media followers and friends. The reception unit can also prioritize receiving relevant voice commands based on the user's social media trends. This makes it possible to analyze the user's social media activity and accept relevant voice input. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit can input the user's social media activity into a generating AI, which can then accept relevant input.

[0045] The analysis unit can adjust the level of detail of the analysis based on the importance of the voice input during analysis. For example, in the case of important voice input, the generating AI performs a detailed analysis. For example, in the case of normal voice input, the generating AI performs a standard analysis. Furthermore, in the case of urgent voice input, the generating AI can perform a rapid analysis. This makes it possible to adjust the level of detail of the analysis based on the importance of the voice input. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the importance of the voice input to the generating AI, and the generating AI can adjust the level of detail of the analysis.

[0046] The analysis unit can apply different analysis algorithms depending on the category of the voice input during analysis. For example, in the case of voice input related to changing smartphone settings, the generating AI will apply an analysis algorithm specialized for setting changes. For example, in the case of voice input related to research, the generating AI will apply an analysis algorithm specialized for information retrieval. Furthermore, in the case of voice input related to e-commerce shopping, the generating AI can apply an analysis algorithm specialized for product search. This makes it possible to apply different analysis algorithms depending on the category of the voice input. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the category of the voice input to the generating AI, and the generating AI can apply an appropriate analysis algorithm.

[0047] The analysis unit can determine the priority of analysis based on the submission date of the voice input during analysis. For example, the analysis unit may prioritize the analysis of recently submitted voice inputs. For example, the analysis unit may determine the priority of analysis based on the deadline specified by the user. The analysis unit may also prioritize the analysis of urgent voice inputs. This makes it possible to determine the priority of analysis based on the submission date of the voice input. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit may input the submission date of the voice input to a generating AI, and the generating AI may determine the priority.

[0048] The analysis unit can adjust the order of analysis based on the relevance of the voice inputs during analysis. For example, the analysis unit may prioritize analyzing voice inputs that are highly relevant. For example, the analysis unit may prioritize analyzing voice inputs that are relevant to the user's current situation. The analysis unit may also prioritize analyzing voice inputs that are highly relevant based on the user's past voice input history. This makes it possible to adjust the order of analysis based on the relevance of the voice inputs. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit may input the relevance of the voice inputs to a generating AI, which can then adjust the order of analysis.

[0049] The processing unit can adjust the level of detail of the processing based on the importance of the voice input during processing. For example, the processing unit performs detailed processing for important voice input. For example, the processing unit performs standard processing for normal voice input. The processing unit can also perform rapid processing for urgent voice input. This makes it possible to adjust the level of detail of the processing based on the importance of the voice input. Some or all of the processing described above in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input the importance of the voice input to a generating AI, and the generating AI can adjust the level of detail of the processing.

[0050] The processing unit can apply different processing algorithms depending on the category of the voice input during processing. For example, in the case of voice input regarding changing smartphone settings, the processing unit applies a processing algorithm specialized for changing settings. For example, in the case of voice input regarding research, the processing unit applies a processing algorithm specialized for information retrieval. Furthermore, in the case of voice input regarding e-commerce shopping, the processing unit can also apply a processing algorithm specialized for product retrieval. This makes it possible to apply different processing algorithms depending on the category of the voice input. Some or all of the processing described above in the processing unit may be performed using AI, for example, or without using AI. For example, the processing unit can input the category of the voice input to a generating AI, and the generating AI can apply an appropriate processing algorithm.

[0051] The processing unit can determine the processing priority based on the timing of voice input submission during processing. For example, the processing unit may prioritize recently submitted voice inputs. For example, the processing unit may determine the processing priority based on the deadline specified by the user. The processing unit may also prioritize urgent voice inputs. This makes it possible to determine the processing priority based on the timing of voice input submission. Some or all of the above processing in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit may input the timing of voice input submission to a generating AI, which can then determine the priority.

[0052] The processing unit can adjust the order of processing based on the relevance of the voice inputs during processing. For example, the processing unit may prioritize processing voice inputs that are highly relevant. For example, the processing unit may prioritize processing voice inputs that are relevant to the user's current situation. The processing unit may also prioritize processing voice inputs that are highly relevant based on the user's past voice input history. This makes it possible to adjust the order of processing based on the relevance of the voice inputs. Some or all of the processing described above in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit may input the relevance of the voice inputs to a generating AI, which can then adjust the order of processing.

[0053] The transfer unit can analyze the user's past device usage history to select the optimal transfer method during the transfer process. For example, the transfer unit can select the optimal transfer method based on the settings of devices the user has used in the past. For example, the transfer unit can prioritize the transfer of specific apps or settings based on the user's past device usage history. The transfer unit can also analyze the user's past device usage patterns and propose the optimal transfer method. This makes it possible to select the optimal transfer method by analyzing the user's past device usage history. Some or all of the above processing in the transfer unit may be performed using AI, for example, or without AI. For example, the transfer unit can input the user's past device usage history into a generating AI, which can then select the optimal transfer method.

[0054] The transfer unit can customize the transfer method based on the user's current device status during the transfer. For example, the transfer unit adjusts the amount of data to be transferred based on the user's current device storage capacity. For example, the transfer unit adjusts the timing of the transfer based on the user's current device battery level. The transfer unit can also select a transfer method based on the user's current device network connection status. This makes it possible to customize the transfer method based on the user's current device status. Some or all of the above processing in the transfer unit may be performed using AI, for example, or without AI. For example, the transfer unit can input the user's current device status to a generating AI, which can then customize the transfer method.

[0055] The handover unit can select the optimal handover method at the time of handover, taking into account the user's geographical location information. For example, if the user is in a specific location, the handover unit can select a handover method suitable for that location. For example, the handover unit can select the optimal handover timing based on the user's current location. Furthermore, if the user is traveling, the handover unit can also select a handover method suitable for the travel destination. This makes it possible to select the optimal handover method while taking into account the user's geographical location information. Some or all of the above processing in the handover unit may be performed using AI, for example, or without using AI. For example, the handover unit can input the user's geographical location information into a generating AI, and the generating AI can select the optimal handover method.

[0056] The handover unit can analyze the user's social media activity and propose a handover method during the handover process. For example, the handover unit can propose the handover of relevant data based on the content of the user's social media posts. For example, the handover unit can propose the handover of relevant data based on the activity of the user's social media followers and friends. The handover unit can also propose the handover of relevant data based on the user's social media trends. This makes it possible to analyze the user's social media activity and propose a handover method. Some or all of the above processing in the handover unit may be performed using AI, for example, or without AI. For example, the handover unit can input the user's social media activity into a generating AI, and the generating AI can propose a handover method.

[0057] The understanding unit can analyze the user's past language usage history to select the optimal understanding method during the understanding process. For example, the understanding unit can select the optimal understanding method based on patterns of language the user has used in the past. For example, the understanding unit can prioritize understanding specific languages ​​or expressions from the user's past language usage history. The understanding unit can also analyze the user's past language usage patterns and propose the optimal understanding method. This makes it possible to analyze the user's past language usage history and select the optimal understanding method. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's past language usage history into a generating AI, which can then select the optimal understanding method.

[0058] The understanding unit can customize its means of understanding based on the user's current language situation during the understanding process. For example, the understanding unit can select the optimal understanding method based on the user's current language settings. For example, the understanding unit can prioritize understanding specific languages ​​or expressions based on the user's current language usage. The understanding unit can also analyze the user's current language usage patterns and suggest the optimal understanding method. This makes it possible to customize the means of understanding based on the user's current language situation. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's current language situation into a generating AI, which can then customize the means of understanding.

[0059] The understanding unit can select the optimal understanding method when understanding, taking into account the user's geographical location information. For example, if the user is in a specific location, the understanding unit can select an understanding method appropriate for that location. For example, the understanding unit can select the optimal understanding method based on the user's current location. Furthermore, if the user is traveling, the understanding unit can also select an understanding method appropriate for the travel destination. This makes it possible to select the optimal understanding method while taking into account the user's geographical location information. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's geographical location information into a generating AI, which can then select the optimal understanding method.

[0060] The understanding unit can analyze the user's social media activity and propose means of understanding during the understanding process. For example, the understanding unit can propose understanding of relevant information based on the content of the user's social media posts. For example, the understanding unit can propose understanding of relevant information based on the activity of the user's social media followers and friends. The understanding unit can also propose understanding of relevant information based on the user's social media trends. This makes it possible to analyze the user's social media activity and propose means of understanding. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's social media activity into a generating AI, and the generating AI can propose means of understanding.

[0061] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0062] The voice assistant agent system can analyze a user's past voice input history and suggest the most appropriate voice commands. For example, it can prioritize suggesting voice commands that the user has frequently used in the past. It can also predict and suggest commands that the user will use at specific times based on their past voice input history. Furthermore, it can analyze the user's voice input patterns and select the most appropriate speech recognition algorithm. This makes it possible to analyze the user's past voice input history and suggest the most appropriate voice commands. For example, if a user has frequently used the command "check the weather" in the past, the system can prioritize suggesting that command. Also, if a user has used the command "check the news" at a specific time, the system can suggest that command at that time. In addition, by analyzing the user's voice input patterns and selecting the most appropriate speech recognition algorithm, the accuracy of speech recognition can be improved.

[0063] A voice assistant agent system can filter voice input based on the user's current situation and areas of interest. For example, it can accept only relevant voice commands based on the user's current situation. It can also prioritize certain voice commands based on the user's areas of interest. Furthermore, it can filter voice commands related to the user's current activity and accept only appropriate commands. This makes it possible to filter voice input based on the user's current situation and areas of interest. For example, if the user is driving, the system can accept only voice commands related to driving. If the user is cooking, the system can prioritize voice commands related to cooking. Furthermore, if the user is watching sports, the system can filter voice commands related to sports and accept only appropriate commands.

[0064] The voice assistant agent system can prioritize receiving voice input that is highly relevant to the user's geographical location. For example, if the user is in a specific location, it can prioritize receiving voice commands related to that location. It can also prioritize receiving voice commands related to nearby information based on the user's current location. Furthermore, if the user is traveling, it can prioritize receiving voice commands related to their travel destination. This makes it possible to prioritize receiving voice input that is highly relevant to the user's geographical location. For example, if the user is in a restaurant, the system can prioritize receiving voice commands related to restaurants. If the user is in a tourist area, the system can prioritize receiving voice commands related to tourist areas. Furthermore, if the user is in an airport, the system can prioritize receiving voice commands related to airports.

[0065] The voice assistant agent system can adjust the level of detail in its analysis based on the importance of the voice input. For example, for important voice input, the generating AI can perform a detailed analysis. For normal voice input, the generating AI can perform a standard analysis. Furthermore, for urgent voice input, the generating AI can perform a rapid analysis. This makes it possible to adjust the level of detail in the analysis based on the importance of the voice input. For example, if a user instructs the system to "look up emergency contacts," the system can perform a rapid analysis and provide the emergency contacts. If a user instructs the system to "look up the latest news," the system can perform a standard analysis and provide the latest news. Furthermore, if a user instructs the system to "create a detailed report," the system can perform a detailed analysis and provide a detailed report.

[0066] The voice assistant agent system can apply different analysis algorithms depending on the category of the voice input during analysis. For example, in the case of voice input regarding changing smartphone settings, the generating AI can apply an analysis algorithm specialized for setting changes. Similarly, in the case of voice input regarding research, the generating AI can apply an analysis algorithm specialized for information retrieval. Furthermore, in the case of voice input regarding e-commerce shopping, the generating AI can apply an analysis algorithm specialized for product retrieval. This makes it possible to apply different analysis algorithms depending on the category of voice input. For example, if the user says, "Turn on Wi-Fi," the system can apply an analysis algorithm specialized for setting changes and turn on Wi-Fi. If the user says, "Research about XX," the system can apply an analysis algorithm specialized for information retrieval and provide information. Furthermore, if the user says, "I want to buy XX," the system can apply an analysis algorithm specialized for product retrieval and provide the product.

[0067] The voice assistant agent system can prioritize analysis based on when the voice input was submitted. For example, it can prioritize the analysis of recently submitted voice inputs. It can also prioritize analysis based on a deadline specified by the user. Furthermore, it can prioritize the analysis of urgent voice inputs. This makes it possible to prioritize analysis based on when the voice input was submitted. For example, if the user instructs, "Please look into this immediately," the system can prioritize the analysis of that voice input. If the user instructs, "Please look into this by tomorrow," the system can prioritize the analysis based on that deadline. Furthermore, if the user instructs, "Please look into this urgent information," the system can prioritize the analysis of that voice input.

[0068] The following briefly describes the processing flow for example form 1.

[0069] Step 1: The reception desk accepts voice input. For example, it accepts voice input when the user speaks into their smartphone. The reception desk can also receive voice input via a microphone. Furthermore, the reception desk can accept various formats of voice input, such as natural language and command formats. Step 2: The analysis unit analyzes the voice input received by the reception unit. For example, it can convert the voice input into text data using speech recognition technology. It can also analyze the content of the voice input using analysis algorithms or generative AI. Step 3: The processing unit performs processing based on the results analyzed by the analysis unit. For example, it can change the settings of a smartphone. It can also search for information on the internet and provide the results, or search for products and proceed with the purchase process. Step 4: The transfer section handles the transfer of settings and apps when changing devices. For example, it retrieves settings and app information from the old device and transfers them to the new device. It can also transfer settings and apps depending on their type. Step 5: The comprehension section handles multilingual support and the understanding of dialects and accents. For example, it supports multiple languages ​​such as English, Japanese, and Chinese. It can also understand the dialects and accents of various regions.

[0070] (Example of form 2) The voice assistant agent system according to an embodiment of the present invention is a system that allows users to perform tasks such as adjusting smartphone settings, searching for information, online shopping, and booking flights and hotels through voice input. This voice assistant agent system can store user information regardless of the smartphone model by linking it to a smartphone model-independent ID (for example, a user account for an application or online service). As a result, user information is carried over even when the user changes models. Furthermore, when switching to a new device, settings and apps from the old device can be transferred to the new device, saving the user the trouble of setting them up again. In addition, this voice assistant agent system supports multiple languages ​​and can understand regional dialects and accents, so it can also be used as an interpreter. For example, the user makes a voice input. For example, the user may make a voice input such as "I want to change my smartphone settings," "I want to search for XX," "I want to buy XX," or "I want to book a flight." This voice input is analyzed by a generating AI and processed appropriately. Next, the generating AI processes tasks such as changing smartphone settings, searching for information, online shopping, and booking flights and hotels based on the voice input. For example, in the case of changing smartphone settings, the generating AI investigates how to change the settings and changes the necessary settings. For research, the generating AI searches the internet for information and provides results. For e-commerce shopping, the generating AI searches for products and completes the purchase process. For flight and hotel reservations, the generating AI searches for reservation sites and completes the reservation process. Furthermore, when changing devices, the generating AI retrieves settings and app information from the old device and transfers it to the new device. This allows the user to use the same settings and apps on the new device. In addition, this voice assistant agent system supports multiple languages ​​and can understand regional dialects and accents. For example, it supports multiple languages ​​such as English, Japanese, and Chinese, and even if the user provides voice input in a different language, the generating AI can appropriately analyze and process it. Furthermore, because it can understand regional dialects and accents, it can also be used as an interpreter.Thus, the voice assistant agent system of the present invention maximizes user convenience by storing user information independently of the device model, handling settings and app transfers when changing devices, and enabling multilingual support and understanding of dialects and accents. As a result, the voice assistant agent system can maximize user convenience.

[0071] The voice assistant agent system according to this embodiment comprises a reception unit, an analysis unit, a processing unit, a handover unit, and an understanding unit. The reception unit receives voice input. The reception unit receives voice input, for example, when a user speaks into a smartphone. The reception unit can also receive voice input through a microphone. Furthermore, the reception unit can accept voice input in various formats, such as natural language or command format. For example, the reception unit receives voice input when a user says, "I want to change the settings on my smartphone." The reception unit can also receive voice input when a user says, "I want to look up XX." Furthermore, the reception unit can also receive voice input when a user says, "I want to buy XX." The analysis unit analyzes the voice input received by the reception unit. The analysis unit converts the voice input into text data, for example, using speech recognition technology. The analysis unit can also analyze the content of the voice input using an analysis algorithm. For example, the analysis unit converts the user's voice input into text data using speech recognition technology. The analysis unit can also analyze the content of the voice input using an analysis algorithm. Furthermore, the analysis unit can also analyze the content of voice input using a generation AI. For example, the analysis unit inputs a prompt to the generation AI, "Please analyze the content of this voice input," and the generation AI analyzes the content of the voice input. The processing unit performs processing based on the results analyzed by the analysis unit. For example, the processing unit can change the settings of the smartphone. The processing unit can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and perform the purchase procedure. For example, the processing unit can change the settings of the smartphone. The processing unit can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and perform the purchase procedure. The transfer unit performs the transfer of settings and applications when changing devices. For example, the transfer unit can acquire settings and application information from the old device and transfer them to the new device. Furthermore, the transfer unit can perform the transfer according to the setting items and application types. For example, the transfer unit can acquire settings and application information from the old device and transfer them to the new device.Furthermore, the transfer unit can perform data transfer depending on the settings and type of application. The understanding unit handles multilingual support and understands dialects and accents. The understanding unit supports multiple languages, such as English, Japanese, and Chinese. It can also understand regional dialects and accents. For example, the understanding unit supports multiple languages, such as English, Japanese, and Chinese. It can also understand regional dialects and accents. As a result, the voice assistant agent system can analyze and process voice input, enabling settings and app transfers when changing devices, multilingual support, and understanding of dialects and accents.

[0072] The reception unit accepts voice input. For example, it accepts voice input when the user speaks into their smartphone. It can also receive voice input via a microphone. Furthermore, the reception unit can accept various voice input formats, including natural language and command formats. For example, it can accept voice input when the user says, "I want to change my smartphone settings." It can also accept voice input when the user says, "I want to look up XX." Furthermore, it can accept voice input when the user says, "I want to buy XX." The reception unit uses a high-sensitivity microphone to accept voice input and incorporates noise-canceling technology to eliminate ambient noise. This allows users to accurately input voice even in noisy environments. In addition, the reception unit can identify the characteristics of the user's voice during voice input and perform personal authentication. This enables the voice assistant agent system to provide customized services for each user. For example, if a user says, "I want to check my calendar," the reception unit recognizes the user's voice and displays the user's calendar information. Furthermore, the reception desk can handle complex requests by using natural language processing technology to understand the user's intent during voice input. For example, if a user says, "Tell me the weather for tomorrow," the reception desk will understand the request and provide appropriate information. This allows the reception desk to respond quickly and accurately to a wide range of user requests.

[0073] The analysis unit analyzes the voice input received by the reception unit. The analysis unit converts the voice input into text data using, for example, speech recognition technology. The analysis unit can also analyze the content of the voice input using analysis algorithms. For example, the analysis unit converts the user's voice input into text data using speech recognition technology. Furthermore, the analysis unit can analyze the content of the voice input using a generative AI. For example, the analysis unit inputs the prompt "Please analyze the content of this voice input" to the generative AI, which then analyzes the content of the voice input. The analysis unit employs a deep learning model as its speech recognition technology, achieving highly accurate speech recognition. This allows it to accurately convert the user's utterance into text data. Furthermore, the analysis unit can analyze the context and intent of the voice input using natural language processing technology. For example, if a user says "Tell me the weather for tomorrow," the analysis unit understands the context and performs appropriate processing to provide weather information. The analysis unit can also analyze the content of the voice input more advancedly using a generative AI. Generative AI learns from vast amounts of data and can deeply understand user utterances. For example, if a user says, "Prepare the materials for the next meeting," the analysis unit can use the generative AI to understand the request and issue instructions for preparing the appropriate materials. This allows the analysis unit to respond quickly and accurately to a wide range of user requests.

[0074] The processing unit performs processing based on the results analyzed by the analysis unit. For example, the processing unit can change the settings of a smartphone. It can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and carry out the purchase process. For example, the processing unit can change the settings of a smartphone. It can also search for information on the internet and provide the results. Furthermore, the processing unit can search for products and carry out the purchase process. The processing unit performs specific actions based on the text data and analysis results received from the analysis unit. For example, if a user says, "Adjust the brightness of my smartphone," the processing unit adjusts the brightness of the smartphone according to that instruction. Also, if a user says, "Find a nearby restaurant," the processing unit searches for information on the internet and provides information on nearby restaurants. Furthermore, if a user says, "I want to buy XX," the processing unit searches for that product and carries out the purchase process. The processing unit can respond flexibly to user requests. For example, if a user says, "Prepare the materials for the next meeting," the processing unit searches for the necessary materials according to that request and prepares them. In addition, the processing unit can provide more personalized services based on the user's past requests and behavioral history. This allows the processing unit to respond quickly and accurately to a wide range of user requests.

[0075] The data transfer unit handles the transfer of settings and apps when changing devices. For example, the data transfer unit retrieves settings and app information from the old device and transfers it to the new device. The data transfer unit can also perform the transfer according to the settings and app types. For example, the data transfer unit retrieves settings and app information from the old device and transfers it to the new device. The data transfer unit automates data transfer from the old device to the new device so that users can smoothly transition to the new device. For example, if a user purchases a new smartphone, the data transfer unit automatically retrieves settings and app information from the old smartphone and transfers it to the new smartphone. This allows the user to use the new device without any hassle. Furthermore, the data transfer unit can select the optimal transfer method according to the settings and app types. For example, it can prioritize the transfer of important settings and app data, and avoid transferring unnecessary data. In addition, the data transfer unit uses encryption technology to protect data in order to ensure data security. This ensures that users' personal information and important data are transferred safely. The data transfer section allows for efficient and secure data transfer, enabling users to smoothly transition to a new device.

[0076] The understanding unit handles multiple languages, including English, Japanese, and Chinese. It can also understand regional dialects and accents. The understanding unit accurately understands and responds appropriately to any language the user speaks in. For example, if a user speaks in English, the understanding unit accurately understands and responds in English. Similarly, if a user speaks in Japanese, the understanding unit accurately understands and responds in Japanese. Furthermore, the understanding unit can understand regional dialects and accents. For example, if a user speaks in Kansai dialect, the understanding unit accurately understands and responds appropriately. The understanding unit combines speech recognition technology and natural language processing technology to achieve multilingual support and dialect / accent understanding. This allows users to speak in their native language or regional dialect, and the voice assistant agent system accurately understands and responds appropriately. The understanding unit can provide a user-friendly system for a wider range of users by supporting diverse languages ​​and dialects.

[0077] The processing unit can change the settings of the smartphone. For example, the processing unit can access the smartphone's settings menu and change the settings according to the user's instructions. For example, if the user instructs the processing unit to "turn on Wi-Fi," the processing unit will turn on the Wi-Fi setting. The processing unit can also adjust the screen brightness if the user instructs it to "adjust the screen brightness." Furthermore, the processing unit can change the notification sound if the user instructs it to "change the notification sound." This makes it possible to change the settings of the smartphone. Some or all of the above processing in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input the user's instructions into a generating AI, which can then analyze and execute the procedure for changing the settings.

[0078] The processing unit can search for information on the internet and provide the results. For example, if the user instructs the processing unit to "research about XX," it will use an internet search engine to search for information. For example, if the user instructs the processing unit to "find the latest news," it will search news sites and provide the latest news. The processing unit can also search recipe sites and provide recipe information if the user instructs it to "find a recipe for XX." Furthermore, if the user instructs it to "find the weather for XX," it can search weather forecast sites and provide weather information. This makes it possible to search for information on the internet and provide the results. Some or all of the above processing in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input the user's instructions into a generating AI, which can then analyze and execute the information retrieval procedure.

[0079] The processing unit can search for products and carry out purchase procedures. For example, if the user instructs the processing unit to "buy XX," it will search e-commerce sites and find the product. For example, if the user instructs the processing unit to "buy a new smartphone," it will search e-commerce sites and provide smartphone product information. The processing unit can also search e-commerce sites for books and provide book product information if the user instructs to "buy a book about XX." Furthermore, if the user instructs to "buy clothes about XX," it can search e-commerce sites for fashion and provide clothing product information. This makes it possible to search for products and carry out purchase procedures. Some or all of the above-described processes in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input user instructions into a generating AI, which can analyze and execute the product search procedure.

[0080] The processing unit can search for reservation websites and perform reservation procedures. For example, if the user instructs the processing unit to "book a flight," it will search for airline reservation websites and perform the reservation procedure. For example, if the user instructs the processing unit to "book a hotel," it will search for hotel reservation websites and perform the reservation procedure. The processing unit can also search for restaurant reservation websites and perform the reservation procedure if the user instructs the processing unit to "book a restaurant." Furthermore, if the user instructs the processing unit to "book a rental car," it will search for rental car reservation websites and perform the reservation procedure. This makes it possible to search for reservation websites and perform reservation procedures. Some or all of the above-described processes in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input user instructions into a generating AI, which can then analyze and execute the reservation procedure steps.

[0081] The transfer unit can acquire settings and application information from the old device and transfer them to the new device. For example, when a user switches to a new smartphone, the transfer unit acquires settings and application information from the old device and transfers them to the new device. For example, the transfer unit acquires Wi-Fi settings, application installation status, user custom settings, etc. from the old device and transfers them to the new device. The transfer unit can also transfer data from applications that the user was using. For example, the transfer unit transfers the application settings and data from the old device to the new device. This makes it possible to transfer settings and application information from the old device to the new device. Some or all of the above processing in the transfer unit may be performed using AI, for example, or without AI. For example, the transfer unit can input the settings and application information from the old device into a generating AI, which can then analyze and execute the transfer procedure.

[0082] The understanding unit supports multiple languages, including English, Japanese, and Chinese, and can understand regional dialects and accents. For example, if a user speaks in English, the understanding unit analyzes the English voice input and processes it appropriately. For example, if a user speaks in Japanese, the understanding unit analyzes the Japanese voice input and processes it appropriately. The understanding unit can also analyze the Chinese voice input if a user speaks in Chinese and process it appropriately. Furthermore, the understanding unit can understand regional dialects and accents. For example, if a user speaks in Kansai dialect, the understanding unit analyzes the Kansai dialect voice input and processes it appropriately. The understanding unit can also understand the accent if a user speaks with a Southern American accent and process it appropriately. This enables multilingual support and the understanding of regional dialects and accents. Some or all of the processing described above in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's voice input into a generating AI, which can handle multilingual support and understand dialects and accents.

[0083] The reception unit can estimate the user's emotions and adjust the timing of voice input reception based on the estimated emotions. For example, if the user is stressed, the reception unit can delay the timing of voice input reception and wait until the user calms down. For example, if the user is relaxed, the reception unit can speed up the timing of voice input reception to receive input smoothly. Also, if the user is in a hurry, the reception unit can immediately receive voice input and start processing quickly. This makes it possible to adjust the timing of voice input reception based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the reception unit may be performed using AI or not using AI. For example, the reception unit can input the user's voice input into the generative AI, which can estimate the emotions and adjust the reception timing.

[0084] The reception unit can analyze the user's past voice input history and select the optimal reception method. For example, the reception unit prioritizes receiving voice commands that the user has frequently used in the past. For example, the reception unit can predict commands to be used during specific time periods based on the user's past voice input history and adjust the reception method accordingly. The reception unit can also analyze the user's voice input patterns and select the optimal speech recognition algorithm. This makes it possible to analyze the user's past voice input history and select the optimal reception method. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit can input the user's past voice input history into a generating AI, which can then select the optimal reception method.

[0085] The reception unit can filter voice input based on the user's current situation and areas of interest. For example, the reception unit may accept only relevant voice commands based on the user's current situation. For example, the reception unit may prioritize accepting specific voice commands based on the user's areas of interest. The reception unit can also filter voice commands related to the user's current activity and accept only appropriate commands. This makes it possible to filter voice input based on the user's current situation and areas of interest. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit may input the user's current situation and areas of interest into a generating AI, which can then perform the filtering.

[0086] The reception unit can estimate the user's emotions and determine the priority of voice input to be received based on the estimated emotions. For example, if the user is stressed, the reception unit will prioritize important voice input. For example, if the user is relaxed, the reception unit will prioritize normal voice input. The reception unit can also prioritize urgent voice input if the user is in a hurry. This makes it possible to determine the priority of voice input based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the reception unit may be performed using AI or not using AI. For example, the reception unit can input the user's voice input into the generative AI, which can estimate the emotions and determine the priority.

[0087] The reception unit can prioritize receiving voice input by considering the user's geographical location information. For example, if the user is in a specific location, the reception unit will prioritize receiving voice commands related to that location. For example, based on the user's current location, the reception unit will prioritize receiving voice commands related to nearby information. Furthermore, if the user is traveling, the reception unit can also prioritize receiving voice commands related to their travel destination. This makes it possible to prioritize receiving voice input that is highly relevant by considering the user's geographical location information. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit can input the user's geographical location information into a generating AI, which can then prioritize receiving input that is highly relevant.

[0088] The reception unit can analyze the user's social media activity and accept relevant input when receiving voice input. For example, the reception unit can prioritize receiving relevant voice commands based on the content of the user's social media posts. For example, the reception unit can prioritize receiving relevant voice commands based on the activity of the user's social media followers and friends. The reception unit can also prioritize receiving relevant voice commands based on the user's social media trends. This makes it possible to analyze the user's social media activity and accept relevant voice input. Some or all of the above processing in the reception unit may be performed using AI, for example, or without AI. For example, the reception unit can input the user's social media activity into a generating AI, which can then accept relevant input.

[0089] The analysis unit can estimate the user's emotions and adjust the expression of the analysis based on the estimated emotions. For example, if the user is stressed, the generation AI will select a concise and easy-to-understand expression. For example, if the user is relaxed, the generation AI will select an expression that includes detailed information. The analysis unit can also select a concise expression if the user is in a hurry. This makes it possible to adjust the expression of the analysis based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the analysis unit may be performed using AI, or not using AI. For example, the analysis unit can input the user's voice input into the generation AI, which can estimate emotions and adjust the expression.

[0090] The analysis unit can adjust the level of detail of the analysis based on the importance of the voice input during analysis. For example, in the case of important voice input, the generating AI performs a detailed analysis. For example, in the case of normal voice input, the generating AI performs a standard analysis. Furthermore, in the case of urgent voice input, the generating AI can perform a rapid analysis. This makes it possible to adjust the level of detail of the analysis based on the importance of the voice input. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the importance of the voice input to the generating AI, and the generating AI can adjust the level of detail of the analysis.

[0091] The analysis unit can apply different analysis algorithms depending on the category of the voice input during analysis. For example, in the case of voice input related to changing smartphone settings, the generating AI will apply an analysis algorithm specialized for setting changes. For example, in the case of voice input related to research, the generating AI will apply an analysis algorithm specialized for information retrieval. Furthermore, in the case of voice input related to e-commerce shopping, the generating AI can apply an analysis algorithm specialized for product search. This makes it possible to apply different analysis algorithms depending on the category of the voice input. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the category of the voice input to the generating AI, and the generating AI can apply an appropriate analysis algorithm.

[0092] The analysis unit can estimate the user's emotions and adjust the length of the analysis based on the estimated emotions. For example, if the user is stressed, the generation AI in the analysis unit will perform a short, concise analysis. For example, if the user is relaxed, the generation AI in the analysis unit will perform a detailed analysis. The analysis unit can also perform a rapid analysis if the user is in a hurry. This makes it possible to adjust the length of the analysis based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's voice input into the generation AI, which will estimate the emotions and adjust the length of the analysis.

[0093] The analysis unit can determine the priority of analysis based on the submission date of the voice input during analysis. For example, the analysis unit may prioritize the analysis of recently submitted voice inputs. For example, the analysis unit may determine the priority of analysis based on the deadline specified by the user. The analysis unit may also prioritize the analysis of urgent voice inputs. This makes it possible to determine the priority of analysis based on the submission date of the voice input. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit may input the submission date of the voice input to a generating AI, and the generating AI may determine the priority.

[0094] The analysis unit can adjust the order of analysis based on the relevance of the voice inputs during analysis. For example, the analysis unit may prioritize analyzing voice inputs that are highly relevant. For example, the analysis unit may prioritize analyzing voice inputs that are relevant to the user's current situation. The analysis unit may also prioritize analyzing voice inputs that are highly relevant based on the user's past voice input history. This makes it possible to adjust the order of analysis based on the relevance of the voice inputs. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit may input the relevance of the voice inputs to a generating AI, which can then adjust the order of analysis.

[0095] The processing unit can estimate the user's emotions and adjust the way the processing is expressed based on the estimated emotions. For example, if the user is stressed, the processing unit will select a concise and easy-to-understand expression. For example, if the user is relaxed, the processing unit will select an expression that includes detailed information. The processing unit can also select a concise expression if the user is in a hurry. This makes it possible to adjust the way the processing is expressed based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the processing described above in the processing unit may be performed using AI, or not using AI. For example, the processing unit can input the user's voice input into the generative AI, which can then estimate the emotions and adjust the expression.

[0096] The processing unit can adjust the level of detail of the processing based on the importance of the voice input during processing. For example, the processing unit performs detailed processing for important voice input. For example, the processing unit performs standard processing for normal voice input. The processing unit can also perform rapid processing for urgent voice input. This makes it possible to adjust the level of detail of the processing based on the importance of the voice input. Some or all of the processing described above in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit can input the importance of the voice input to a generating AI, and the generating AI can adjust the level of detail of the processing.

[0097] The processing unit can apply different processing algorithms depending on the category of the voice input during processing. For example, in the case of voice input regarding changing smartphone settings, the processing unit applies a processing algorithm specialized for changing settings. For example, in the case of voice input regarding research, the processing unit applies a processing algorithm specialized for information retrieval. Furthermore, in the case of voice input regarding e-commerce shopping, the processing unit can also apply a processing algorithm specialized for product retrieval. This makes it possible to apply different processing algorithms depending on the category of the voice input. Some or all of the processing described above in the processing unit may be performed using AI, for example, or without using AI. For example, the processing unit can input the category of the voice input to a generating AI, and the generating AI can apply an appropriate processing algorithm.

[0098] The processing unit can estimate the user's emotions and adjust the length of the processing based on the estimated emotions. For example, if the user is stressed, the processing unit will perform short, concise processing. For example, if the user is relaxed, the processing unit will perform detailed processing. The processing unit can also perform rapid processing if the user is in a hurry. This makes it possible to adjust the length of the processing based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the processing described above in the processing unit may be performed using AI or not using AI. For example, the processing unit can input the user's voice input into the generative AI, which can then estimate the emotions and adjust the length of the processing.

[0099] The processing unit can determine the processing priority based on the timing of voice input submission during processing. For example, the processing unit may prioritize recently submitted voice inputs. For example, the processing unit may determine the processing priority based on the deadline specified by the user. The processing unit may also prioritize urgent voice inputs. This makes it possible to determine the processing priority based on the timing of voice input submission. Some or all of the above processing in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit may input the timing of voice input submission to a generating AI, which can then determine the priority.

[0100] The processing unit can adjust the order of processing based on the relevance of the voice inputs during processing. For example, the processing unit may prioritize processing voice inputs that are highly relevant. For example, the processing unit may prioritize processing voice inputs that are relevant to the user's current situation. The processing unit may also prioritize processing voice inputs that are highly relevant based on the user's past voice input history. This makes it possible to adjust the order of processing based on the relevance of the voice inputs. Some or all of the processing described above in the processing unit may be performed using AI, for example, or without AI. For example, the processing unit may input the relevance of the voice inputs to a generating AI, which can then adjust the order of processing.

[0101] The handover unit can estimate the user's emotions and adjust the handover method based on the estimated emotions. For example, if the user is stressed, the handover unit will select a concise and easy-to-understand handover method. For example, if the user is relaxed, the handover unit will select a handover method that includes detailed information. The handover unit can also select a quick handover method if the user is in a hurry. This makes it possible to adjust the handover method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the handover unit may be performed using AI or not using AI. For example, the handover unit can input the user's voice input into the generative AI, which will estimate the emotions and adjust the handover method.

[0102] The transfer unit can analyze the user's past device usage history to select the optimal transfer method during the transfer process. For example, the transfer unit can select the optimal transfer method based on the settings of devices the user has used in the past. For example, the transfer unit can prioritize the transfer of specific apps or settings based on the user's past device usage history. The transfer unit can also analyze the user's past device usage patterns and propose the optimal transfer method. This makes it possible to select the optimal transfer method by analyzing the user's past device usage history. Some or all of the above processing in the transfer unit may be performed using AI, for example, or without AI. For example, the transfer unit can input the user's past device usage history into a generating AI, which can then select the optimal transfer method.

[0103] The transfer unit can customize the transfer method based on the user's current device status during the transfer. For example, the transfer unit adjusts the amount of data to be transferred based on the user's current device storage capacity. For example, the transfer unit adjusts the timing of the transfer based on the user's current device battery level. The transfer unit can also select a transfer method based on the user's current device network connection status. This makes it possible to customize the transfer method based on the user's current device status. Some or all of the above processing in the transfer unit may be performed using AI, for example, or without AI. For example, the transfer unit can input the user's current device status to a generating AI, which can then customize the transfer method.

[0104] The handover unit can estimate the user's emotions and determine the priority of the handover based on the estimated emotions. For example, if the user is stressed, the handover unit will prioritize the handover of important data. For example, if the user is relaxed, the handover unit will prioritize the handover of normal data. The handover unit can also prioritize the handover of urgent data if the user is in a hurry. This makes it possible to determine the priority of the handover based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the handover unit may be performed using AI or not using AI. For example, the handover unit can input the user's voice input into the generative AI, which will estimate the emotions and determine the priority of the handover.

[0105] The handover unit can select the optimal handover method at the time of handover, taking into account the user's geographical location information. For example, if the user is in a specific location, the handover unit can select a handover method suitable for that location. For example, the handover unit can select the optimal handover timing based on the user's current location. Furthermore, if the user is traveling, the handover unit can also select a handover method suitable for the travel destination. This makes it possible to select the optimal handover method while taking into account the user's geographical location information. Some or all of the above processing in the handover unit may be performed using AI, for example, or without using AI. For example, the handover unit can input the user's geographical location information into a generating AI, and the generating AI can select the optimal handover method.

[0106] The handover unit can analyze the user's social media activity and propose a handover method during the handover process. For example, the handover unit can propose the handover of relevant data based on the content of the user's social media posts. For example, the handover unit can propose the handover of relevant data based on the activity of the user's social media followers and friends. The handover unit can also propose the handover of relevant data based on the user's social media trends. This makes it possible to analyze the user's social media activity and propose a handover method. Some or all of the above processing in the handover unit may be performed using AI, for example, or without AI. For example, the handover unit can input the user's social media activity into a generating AI, and the generating AI can propose a handover method.

[0107] The understanding unit can estimate the user's emotions and adjust its understanding method based on the estimated emotions. For example, if the user is stressed, the understanding unit will select a concise and easy-to-understand understanding method. For example, if the user is relaxed, the understanding unit will select an understanding method that includes detailed information. The understanding unit can also select a concise understanding method if the user is in a hurry. This makes it possible to adjust the understanding method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's voice input into the generative AI, which will estimate the emotions and adjust the understanding method.

[0108] The understanding unit can analyze the user's past language usage history to select the optimal understanding method during the understanding process. For example, the understanding unit can select the optimal understanding method based on patterns of language the user has used in the past. For example, the understanding unit can prioritize understanding specific languages ​​or expressions from the user's past language usage history. The understanding unit can also analyze the user's past language usage patterns and propose the optimal understanding method. This makes it possible to analyze the user's past language usage history and select the optimal understanding method. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's past language usage history into a generating AI, which can then select the optimal understanding method.

[0109] The understanding unit can customize its means of understanding based on the user's current language situation during the understanding process. For example, the understanding unit can select the optimal understanding method based on the user's current language settings. For example, the understanding unit can prioritize understanding specific languages ​​or expressions based on the user's current language usage. The understanding unit can also analyze the user's current language usage patterns and suggest the optimal understanding method. This makes it possible to customize the means of understanding based on the user's current language situation. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's current language situation into a generating AI, which can then customize the means of understanding.

[0110] The understanding unit can estimate the user's emotions and determine the priority of understanding based on the estimated emotions. For example, if the user is stressed, the understanding unit will prioritize understanding important information. For example, if the user is relaxed, the understanding unit will prioritize understanding normal information. The understanding unit can also prioritize understanding urgent information if the user is in a hurry. This makes it possible to determine the priority of understanding based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the understanding unit may be performed using AI, or not using AI. For example, the understanding unit can input the user's voice input into the generative AI, which can estimate emotions and determine the priority of understanding.

[0111] The understanding unit can select the optimal understanding method when understanding, taking into account the user's geographical location information. For example, if the user is in a specific location, the understanding unit can select an understanding method appropriate for that location. For example, the understanding unit can select the optimal understanding method based on the user's current location. Furthermore, if the user is traveling, the understanding unit can also select an understanding method appropriate for the travel destination. This makes it possible to select the optimal understanding method while taking into account the user's geographical location information. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's geographical location information into a generating AI, which can then select the optimal understanding method.

[0112] The understanding unit can analyze the user's social media activity and propose means of understanding during the understanding process. For example, the understanding unit can propose understanding of relevant information based on the content of the user's social media posts. For example, the understanding unit can propose understanding of relevant information based on the activity of the user's social media followers and friends. The understanding unit can also propose understanding of relevant information based on the user's social media trends. This makes it possible to analyze the user's social media activity and propose means of understanding. Some or all of the above processing in the understanding unit may be performed using AI, for example, or without AI. For example, the understanding unit can input the user's social media activity into a generating AI, and the generating AI can propose means of understanding.

[0113] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0114] A voice assistant agent system can estimate a user's emotions and adjust the content of voice input based on those emotions. For example, if a user is stressed, the system can provide a relaxing voice guide. If the user is relaxed, the system can provide detailed information. Furthermore, if the user is in a hurry, the system can provide concise information that gets straight to the point. This enables the provision of appropriate information according to the user's emotions. Emotion estimation is achieved using an emotion engine or generative AI. For example, generative AI can analyze the user's voice input and estimate their emotions. Based on the estimated emotions, the system can adjust the content of the voice input and provide the user with the most suitable information.

[0115] The voice assistant agent system can analyze a user's past voice input history and suggest the most appropriate voice commands. For example, it can prioritize suggesting voice commands that the user has frequently used in the past. It can also predict and suggest commands that the user will use at specific times based on their past voice input history. Furthermore, it can analyze the user's voice input patterns and select the most appropriate speech recognition algorithm. This makes it possible to analyze the user's past voice input history and suggest the most appropriate voice commands. For example, if a user has frequently used the command "check the weather" in the past, the system can prioritize suggesting that command. Also, if a user has used the command "check the news" at a specific time, the system can suggest that command at that time. In addition, by analyzing the user's voice input patterns and selecting the most appropriate speech recognition algorithm, the accuracy of speech recognition can be improved.

[0116] A voice assistant agent system can filter voice input based on the user's current situation and areas of interest. For example, it can accept only relevant voice commands based on the user's current situation. It can also prioritize certain voice commands based on the user's areas of interest. Furthermore, it can filter voice commands related to the user's current activity and accept only appropriate commands. This makes it possible to filter voice input based on the user's current situation and areas of interest. For example, if the user is driving, the system can accept only voice commands related to driving. If the user is cooking, the system can prioritize voice commands related to cooking. Furthermore, if the user is watching sports, the system can filter voice commands related to sports and accept only appropriate commands.

[0117] A voice assistant agent system can estimate a user's emotions and prioritize voice input based on those emotions. For example, if a user is stressed, important voice input can be prioritized. If a user is relaxed, normal voice input can be prioritized. Furthermore, if a user is in a hurry, urgent voice input can be prioritized. This makes it possible to prioritize voice input based on the user's emotions. Emotion estimation is achieved using an emotion engine or generative AI. For example, generative AI can analyze the user's voice input and estimate their emotions. Based on the estimated emotions, the system can prioritize voice input and prioritize important voice input.

[0118] The voice assistant agent system can prioritize receiving voice input that is highly relevant to the user's geographical location. For example, if the user is in a specific location, it can prioritize receiving voice commands related to that location. It can also prioritize receiving voice commands related to nearby information based on the user's current location. Furthermore, if the user is traveling, it can prioritize receiving voice commands related to their travel destination. This makes it possible to prioritize receiving voice input that is highly relevant to the user's geographical location. For example, if the user is in a restaurant, the system can prioritize receiving voice commands related to restaurants. If the user is in a tourist area, the system can prioritize receiving voice commands related to tourist areas. Furthermore, if the user is in an airport, the system can prioritize receiving voice commands related to airports.

[0119] A voice assistant agent system can estimate the user's emotions and adjust the presentation of its analysis based on those emotions. For example, if the user is stressed, the generating AI can select a concise and easy-to-understand presentation. If the user is relaxed, the generating AI can select a presentation that includes detailed information. Furthermore, if the user is in a hurry, the generating AI can select a presentation that gets straight to the point. This makes it possible to adjust the presentation of the analysis based on the user's emotions. Emotion estimation is achieved using an emotion engine or generating AI. For example, the generating AI can analyze the user's voice input and estimate their emotions. Based on the estimated emotions, the system can adjust the presentation of its analysis and provide the user with the most relevant information.

[0120] The voice assistant agent system can adjust the level of detail in its analysis based on the importance of the voice input. For example, for important voice input, the generating AI can perform a detailed analysis. For normal voice input, the generating AI can perform a standard analysis. Furthermore, for urgent voice input, the generating AI can perform a rapid analysis. This makes it possible to adjust the level of detail in the analysis based on the importance of the voice input. For example, if a user instructs the system to "look up emergency contacts," the system can perform a rapid analysis and provide the emergency contacts. If a user instructs the system to "look up the latest news," the system can perform a standard analysis and provide the latest news. Furthermore, if a user instructs the system to "create a detailed report," the system can perform a detailed analysis and provide a detailed report.

[0121] The voice assistant agent system can apply different analysis algorithms depending on the category of the voice input during analysis. For example, in the case of voice input regarding changing smartphone settings, the generating AI can apply an analysis algorithm specialized for setting changes. Similarly, in the case of voice input regarding research, the generating AI can apply an analysis algorithm specialized for information retrieval. Furthermore, in the case of voice input regarding e-commerce shopping, the generating AI can apply an analysis algorithm specialized for product retrieval. This makes it possible to apply different analysis algorithms depending on the category of voice input. For example, if the user says, "Turn on Wi-Fi," the system can apply an analysis algorithm specialized for setting changes and turn on Wi-Fi. If the user says, "Research about XX," the system can apply an analysis algorithm specialized for information retrieval and provide information. Furthermore, if the user says, "I want to buy XX," the system can apply an analysis algorithm specialized for product retrieval and provide the product.

[0122] Voice assistant agent systems can estimate a user's emotions and adjust the length of their analysis based on that estimation. For example, if a user is stressed, the generating AI can produce a short, concise analysis. If the user is relaxed, the generating AI can produce a more detailed analysis. Furthermore, if the user is in a hurry, the generating AI can produce a rapid analysis. This allows the system to adjust the length of the analysis based on the user's emotions. Emotion estimation is achieved using an emotion engine or generating AI. For example, a generating AI can analyze the user's voice input and estimate their emotions. Based on the estimated emotions, the system can adjust the length of the analysis to provide the user with the most relevant information.

[0123] The voice assistant agent system can prioritize analysis based on when the voice input was submitted. For example, it can prioritize the analysis of recently submitted voice inputs. It can also prioritize analysis based on a deadline specified by the user. Furthermore, it can prioritize the analysis of urgent voice inputs. This makes it possible to prioritize analysis based on when the voice input was submitted. For example, if the user instructs, "Please look into this immediately," the system can prioritize the analysis of that voice input. If the user instructs, "Please look into this by tomorrow," the system can prioritize the analysis based on that deadline. Furthermore, if the user instructs, "Please look into this urgent information," the system can prioritize the analysis of that voice input.

[0124] The following briefly describes the processing flow for example form 2.

[0125] Step 1: The reception desk accepts voice input. For example, it accepts voice input when the user speaks into their smartphone. The reception desk can also receive voice input via a microphone. Furthermore, the reception desk can accept various formats of voice input, such as natural language and command formats. Step 2: The analysis unit analyzes the voice input received by the reception unit. For example, it can convert the voice input into text data using speech recognition technology. It can also analyze the content of the voice input using analysis algorithms or generative AI. Step 3: The processing unit performs processing based on the results analyzed by the analysis unit. For example, it can change the settings of a smartphone. It can also search for information on the internet and provide the results, or search for products and proceed with the purchase process. Step 4: The transfer section handles the transfer of settings and apps when changing devices. For example, it retrieves settings and app information from the old device and transfers them to the new device. It can also transfer settings and apps depending on their type. Step 5: The comprehension section handles multilingual support and the understanding of dialects and accents. For example, it supports multiple languages ​​such as English, Japanese, and Chinese. It can also understand the dialects and accents of various regions.

[0126] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0127] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0128] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0129] Each of the multiple elements described above, including the reception unit, analysis unit, processing unit, handover unit, and understanding unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the reception unit receives voice input using the microphone 38B of the smart device 14 and processes the voice data with the control unit 46A. The analysis unit converts the voice input into text data using the specific processing unit 290 of the data processing unit 12 and analyzes the content using an analysis algorithm. The processing unit performs tasks such as changing smartphone settings, internet searches, and e-commerce shopping using the specific processing unit 290 of the data processing unit 12. The handover unit acquires settings and application information from the old device using the specific processing unit 290 of the data processing unit 12 and transfers it to the new device. The understanding unit performs multilingual support and dialect / accent understanding using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be changed in various ways.

[0130] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0131] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0132] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0133] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0134] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0135] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0136] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0137] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0138] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0139] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0140] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0141] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0142] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0143] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0144] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0145] Each of the multiple elements described above, including the reception unit, analysis unit, processing unit, handover unit, and understanding unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the reception unit receives voice input using the microphone 238 of the smart glasses 214 and processes the voice data with the control unit 46A. The analysis unit converts the voice input into text data using the specific processing unit 290 of the data processing unit 12 and analyzes the content using an analysis algorithm. The processing unit performs tasks such as changing smartphone settings, internet searches, and e-commerce shopping using the specific processing unit 290 of the data processing unit 12. The handover unit acquires settings and application information from the old device using the specific processing unit 290 of the data processing unit 12 and transfers it to the new device. The understanding unit performs multilingual support and dialect / accent understanding using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0146] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0147] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0148] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0149] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0150] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0151] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0152] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0153] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0154] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0155] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0156] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0157] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0158] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0159] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0160] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0161] Each of the multiple elements described above, including the reception unit, analysis unit, processing unit, handover unit, and understanding unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the reception unit receives voice input using the microphone 238 of the headset terminal 314 and processes the voice data with the control unit 46A. The analysis unit converts the voice input into text data using the specific processing unit 290 of the data processing unit 12 and analyzes the content using an analysis algorithm. The processing unit performs tasks such as changing smartphone settings, internet searches, and e-commerce shopping using the specific processing unit 290 of the data processing unit 12. The handover unit acquires settings and application information from the old device using the specific processing unit 290 of the data processing unit 12 and transfers it to the new device. The understanding unit performs multilingual support and dialect / accent understanding using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0162] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0163] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0164] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0165] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0166] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0167] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0168] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0169] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0170] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0171] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0172] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0173] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0174] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0175] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0176] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0177] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0178] Each of the multiple elements described above, including the reception unit, analysis unit, processing unit, handover unit, and understanding unit, is implemented, for example, by at least one of the robot 414 and the data processing unit 12. For example, the reception unit receives voice input using the microphone 238 of the robot 414 and processes the voice data with the control unit 46A. The analysis unit converts the voice input into text data using the specific processing unit 290 of the data processing unit 12 and analyzes the content using an analysis algorithm. The processing unit performs tasks such as changing smartphone settings, internet searches, and e-commerce shopping using the specific processing unit 290 of the data processing unit 12. The handover unit acquires settings and application information from the old device using the specific processing unit 290 of the data processing unit 12 and transfers it to the new device. The understanding unit performs multilingual support and dialect / accent understanding using the specific processing unit 290 of the data processing unit 12. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be changed in various ways.

[0179] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0180] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0181] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0182] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0183] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0184] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0185] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0186] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0187] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0188] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0189] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0190] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0191] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0192] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0193] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0194] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0195] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0196] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0197] (Note 1) A reception desk that accepts voice input, An analysis unit analyzes the voice input received by the reception unit, A processing unit that performs processing based on the results of the analysis performed by the analysis unit, The transfer section handles settings and app transfers when changing devices, It includes an understanding unit that handles multiple languages ​​and understands dialects and accents. A system characterized by the following features. (Note 2) The aforementioned processing unit, Change your smartphone settings The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned processing unit, Search the internet for information and provide the results. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned processing unit, Search for products and proceed with the purchase. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned processing unit, Search for a booking website and proceed with the booking process. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned handover section is, Retrieve settings and app information from the old device and transfer them to the new device. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned understanding unit is, It supports multiple languages, including English, Japanese, and Chinese, and understands regional dialects and accents. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned reception unit is The system estimates the user's emotions and adjusts the timing of voice input acceptance based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned reception unit is The system analyzes the user's past voice input history and selects the optimal reception method. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned reception unit is When receiving voice input, filtering is performed based on the user's current situation and areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned reception unit is It estimates the user's emotions and determines the priority of voice input to accept based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned reception unit is When receiving voice input, the system prioritizes accepting input that is highly relevant, taking into account the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned reception unit is When receiving voice input, the system analyzes the user's social media activity and accepts relevant input. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned analysis unit, The system estimates the user's emotions and adjusts the representation of the analysis based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned analysis unit, During analysis, the level of detail of the analysis is adjusted based on the importance of the voice input. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned analysis unit, During analysis, different analysis algorithms are applied depending on the category of the audio input. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned analysis unit, It estimates the user's emotions and adjusts the length of the analysis based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned analysis unit, During analysis, the priority of analysis is determined based on when the voice input was submitted. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned analysis unit, During analysis, the order of analysis is adjusted based on the relevance of the audio input. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned processing unit, It estimates the user's emotions and adjusts the way processing is represented based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned processing unit, During processing, adjust the level of detail based on the importance of the voice input. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned processing unit, During processing, different processing algorithms are applied depending on the category of the voice input. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned processing unit, It estimates the user's emotions and adjusts the processing length based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned processing unit, During processing, the processing priority is determined based on when the voice input was submitted. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned processing unit, During processing, the order of processing is adjusted based on the relevance of the voice input. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned handover section is, It estimates the user's emotions and adjusts the handover method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned handover section is, During the data transfer process, the system analyzes the user's past device usage history to select the optimal transfer method. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned handover section is, During the transfer process, the transfer method is customized based on the user's current device status. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned handover section is, It estimates the user's emotions and determines the priority of the handover based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned handover section is, During the data transfer process, the optimal transfer method will be selected, taking into account the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned handover section is, During the handover process, we analyze the user's social media activity and propose handover methods. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned understanding unit is, It estimates the user's emotions and adjusts its understanding based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned understanding unit is, During comprehension, the system analyzes the user's past language usage history to select the optimal comprehension method. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned understanding unit is, When comprehending, the means of comprehension are customized based on the user's current language status. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned understanding unit is, It estimates the user's emotions and determines the priority of understanding based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned understanding unit is, When understanding data, the system selects the optimal understanding method by considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 37) The aforementioned understanding unit is, During the understanding process, we analyze users' social media activity and propose methods for understanding. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0198] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A reception desk that accepts voice input, An analysis unit analyzes the voice input received by the reception unit, A processing unit that performs processing based on the results of analysis by the aforementioned analysis unit, The transfer section handles settings and app transfers when changing devices, It includes an understanding unit that handles multiple languages ​​and understands dialects and accents. A system characterized by the following features.

2. The aforementioned processing unit, Change your smartphone settings The system according to feature 1.

3. The aforementioned processing unit, Search the internet for information and provide the results. The system according to feature 1.

4. The aforementioned processing unit, Search for products and proceed with the purchase. The system according to feature 1.

5. The aforementioned processing unit, Search for a booking website and proceed with the booking process. The system according to feature 1.

6. The aforementioned handover unit is, Retrieve settings and app information from the old device and transfer them to the new device. The system according to feature 1.

7. The aforementioned understanding unit is, It supports multiple languages, including English, Japanese, and Chinese, and understands regional dialects and accents. The system according to feature 1.

8. The aforementioned reception unit is The system estimates the user's emotions and adjusts the timing of voice input acceptance based on the estimated emotions. The system according to feature 1.

Images