system

Abstract

The system according to this embodiment aims to automate responses to visitors and ensure the safety and comfort of residents. [Solution] The system according to this embodiment comprises a recognition unit, a collection unit, a response unit, and a provision unit. The recognition unit recognizes visitors. The collection unit collects information about visitors recognized by the recognition unit. The response unit responds to visitors based on the information collected by the collection unit. The provision unit provides the response generated by the response unit.

Description

Technical Field

[0006] , , ,

[0005] , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0007] The system according to this embodiment can automate responses to visitors and ensure the safety and comfort of residents. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 4, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The AI ​​agent reception system according to an embodiment of the present invention is a system that handles reception duties for visitors approaching a home and telephone calls, ensuring the safety and comfort of residents. This AI agent reception system supports the elderly, children, and people with disabilities by having an AI agent handle reception duties for visitors and telephone calls, eliminating inconvenience and discomfort for users and helping to avoid trouble. For example, when a visitor approaches the door, a multimodal AI camera recognizes the visitor, and the AI ​​agent inquires about the visitor's purpose through a microphone. The generating AI analyzes the information and generates an appropriate response. For example, in the case of door-to-door sales or solicitations, the AI ​​agent will handle the refusal. In the case of deliveries, the resident can receive the package by displaying a QR code (registered trademark). If the resident is absent, the AI ​​agent will prompt the visitor to return. Furthermore, the AI ​​agent also handles telephone calls, recording the conversation using Slack transcription and recording functions. This allows residents to check the status of the interaction in real time. In addition, visit images are recorded and can be viewed and shared on indoor monitors or devices. This system is particularly effective for the elderly, children, people with disabilities, and women living alone, ensuring the safety and comfort of residents by supporting them in dealing with visitors. For example, it provides support tailored to individual needs, such as transcribing spoken words into text and creating example responses for the hearing impaired, and providing voice descriptions of the visitor's characteristics for the visually impaired. In this way, introducing an AI agent reception system ensures the safety and comfort of residents and can cope with the increase in teleworking, online shopping, and services like Uber in the new normal society. It also reduces the risk of crime and trouble from visitors, providing peace of mind to residents. Thus, the AI ​​agent reception system can ensure the safety and comfort of residents.

[0029] The AI ​​agent reception system according to this embodiment comprises a recognition unit, a collection unit, a response unit, and a provision unit. The recognition unit recognizes visitors. The recognition unit can recognize visitors using, for example, facial recognition technology. The recognition unit can also recognize visitors using speech recognition technology. Furthermore, the recognition unit can also recognize visitors using motion recognition technology. For example, the recognition unit uses facial recognition technology to photograph the visitor's face with a camera and recognizes it by comparing it with a database. When using speech recognition technology, the recognition unit captures the visitor's voice with a microphone and recognizes it by analyzing the audio data. When using motion recognition technology, the recognition unit photographs the visitor's movements with a camera and recognizes them by analyzing the movement patterns. The collection unit collects information about visitors recognized by the recognition unit. For example, the collection unit can collect the visitor's name and purpose of visit. Furthermore, the collection unit can also collect the visitor's purpose of visit and visit time. Furthermore, the collection unit can also collect the visitor's contact information. For example, the collection unit asks the visitor for their name and collects it as audio data. The system inquires about the visitor's purpose and collects this information as text data. It also inquires about the visitor's purpose and visit duration and stores this information in a database. Furthermore, it inquires about the visitor's contact information and registers it in the contact database. The response unit responds to the visitor based on the information collected by the collection unit. For example, the response unit can respond according to the visitor's purpose. It can also respond according to the visitor's purpose. Furthermore, it can respond according to the visitor's visit duration. For example, the response unit provides appropriate responses according to the visitor's purpose. It provides necessary responses according to the visitor's purpose. It provides prompt responses according to the visitor's visit duration. The delivery unit provides the responses generated by the response unit. For example, the delivery unit can provide responses via voice. It can also provide responses via text. Furthermore, it can provide responses via video call. For example, the delivery unit can provide responses via voice using speech synthesis technology. It can provide responses via text using text generation technology. It can provide responses via video call using video call technology. As a result, the AI ​​agent reception system according to this embodiment can consistently perform tasks from recognizing visitors to responding to them and providing answers.

[0030] The recognition unit recognizes visitors. For example, the recognition unit can recognize visitors using facial recognition technology. Specifically, it uses a camera to capture the visitor's face and inputs the image data into a facial recognition algorithm. The facial recognition algorithm compares the image with a pre-registered facial database to identify the visitor. Facial recognition technology includes methods using feature point extraction and deep learning, which enables highly accurate recognition. The recognition unit can also recognize visitors using speech recognition technology. In speech recognition technology, a microphone is used to capture the visitor's voice and inputs the audio data into a speech recognition engine. The speech recognition engine analyzes the audio data, identifies speech patterns, and converts them into text data. This allows the recognition unit to recognize the visitor's name and purpose from their voice. Furthermore, the recognition unit can also recognize visitors using motion recognition technology. In motion recognition technology, a camera is used to capture the visitor's movements and inputs the video data into a motion recognition algorithm. The motion recognition algorithm analyzes the visitor's movement patterns and identifies specific movements. For example, if a visitor waves or makes a specific gesture, the system can recognize the action and respond appropriately. This allows the recognition unit to combine various recognition technologies, including facial recognition, voice recognition, and motion recognition, to recognize the visitor from multiple perspectives.

[0031] The collection unit collects information about visitors recognized by the recognition unit. For example, the collection unit can collect the visitor's name and purpose of visit. Specifically, it asks the visitor for their name and collects the answer as voice data. The voice data is converted into text data by a speech recognition engine and stored in a database. The collection unit also asks the visitor for their purpose of visit and collects the answer as text data. The visitor's purpose of visit is collected either by converting voice to text using speech recognition technology or by the visitor entering it into a tablet or kiosk terminal. Furthermore, the collection unit can also collect the visitor's purpose of visit and visit duration. The purpose of visit is collected by asking the visitor a question and storing the answer in a database. The visit duration is collected by automatically recording the time the visitor accessed the reception system. The collection unit centrally manages this information and can link with other systems and departments as needed. For example, the collected information is stored on a cloud server and made accessible to the response and service departments. The collection unit can also collect visitor contact information. Contact information is collected by asking visitors for their phone number and email address and registering their responses in a database. This allows the data collection unit to efficiently gather detailed visitor information and improve the overall system performance.

[0032] The response department responds to visitors based on the information collected by the data collection department. For example, the response department can respond according to the visitor's request. Specifically, it can contact the appropriate person or guide the visitor through the necessary procedures, depending on the visitor's request. The response department can also respond according to the purpose of the visitor's visit. For example, if a visitor is visiting a specific department or person, it will notify that department or person of the visitor's information and request their assistance. Furthermore, the response department can respond according to the visitor's visit time. For example, if the visit time is scheduled in advance, it will prepare for the response according to that time and provide a prompt response. Based on the collected information, the response department can provide flexible responses according to the visitor's needs. For example, if a visitor has an urgent matter, it will be given priority. Also, if a visitor requests a specific service, it will provide information about that service and guide the visitor through the necessary procedures. The response department is required to provide prompt and appropriate responses in order to increase visitor satisfaction. This allows the response department to provide flexible responses according to visitor needs and improve the reliability and efficiency of the entire system.

[0033] The service provider provides the response generated by the response provider. The service provider can provide the response, for example, by voice. Specifically, it can use speech synthesis technology to provide the response generated by the response provider in voice. Speech synthesis technology converts text data into voice data and transmits it to the visitor through a speaker. The service provider can also provide the response in text. It can use text generation technology to provide the response generated by the response provider as text data. The text data can be displayed on a screen or tablet device and communicated visually to the visitor. Furthermore, the service provider can also provide the response via video call. It can use video call technology to provide the response generated by the response provider via video call. Video call technology uses a camera and microphone to communicate with the visitor in real time. This allows the service provider to provide responses to visitors in a variety of ways. For example, if a voice response is appropriate, it will be provided by voice; if a text response is appropriate, it will be provided by text; and if a video call is necessary, it will be handled by video call. This allows the service provider to provide responses in the most appropriate way according to the visitor's needs and increase visitor satisfaction. Furthermore, the service department can collect feedback from visitors and continuously improve the accuracy and effectiveness of its responses. This allows the service department to provide visitors with quick and appropriate answers, thereby improving the overall reliability and efficiency of the system.

[0034] The recognition unit can recognize visitors using a multimodal AI camera. For example, the recognition unit can recognize the visitor's face and actions using the multimodal AI camera. For example, the recognition unit can capture the visitor's face using the multimodal AI camera and recognize it using face recognition technology. The recognition unit can also capture the visitor's actions using the multimodal AI camera and recognize them using action recognition technology. Furthermore, the recognition unit can capture the visitor's voice using the multimodal AI camera and recognize it using voice recognition technology. As a result, the accuracy of visitor recognition is improved by using a multimodal AI camera. A multimodal AI camera can, for example, combine multiple sensors to simultaneously recognize multiple modals such as faces, actions, and voices. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input data acquired by the multimodal AI camera into a generative AI and have the generative AI perform visitor recognition.

[0035] The data collection unit can collect visitor requests using a sound-collecting microphone. For example, the data collection unit can use a sound-collecting microphone to capture the visitor's voice and collect their request. For example, the data collection unit can use a directional microphone to capture the visitor's voice, remove background noise, and collect their request. Alternatively, the data collection unit can use an omnidirectional microphone to capture a wide range of sounds, identify the visitor's voice, and collect their request. Furthermore, the data collection unit can use a sound-collecting microphone to analyze the visitor's voice tone and speaking style and collect their request. This allows for accurate collection of visitor requests by using a sound-collecting microphone. There are various types of sound-collecting microphones, such as high-sensitivity microphones and noise-canceling microphones, which can be selected according to the application. Some or all of the above-described processing in the data collection unit may be performed using, for example, a generative AI, or without a generative AI. For example, the data collection unit can input the audio data acquired by the sound-collecting microphone into a generative AI and have the generative AI collect visitor requests.

[0036] The response unit can generate answers to visitors' requests using a generative AI. For example, the response unit can use a generative AI to generate answers that are appropriate to the visitor's request. For example, the response unit inputs the visitor's request into the generative AI, and the generative AI generates an appropriate answer. The generative AI can use, for example, a text generation AI (e.g., LLM) to generate answers to visitors' requests. The generative AI can also use a multimodal generative AI to generate answers to visitors' requests. Furthermore, the generative AI can generate answers in formats such as voice, text, and video, depending on the visitor's request. Thus, by using the generative AI, appropriate answers to visitors' requests can be generated. The generative AI has, for example, learned from a large amount of data and possesses advanced natural language processing capabilities. The generative AI can generate the optimal answer according to the visitor's request. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the response unit can input the visitor's request into the generative AI and provide the answer generated by the generative AI.

[0037] The service provider can provide the generated response to the visitor. The service provider can provide the response, for example, by voice. For example, the service provider can provide the generated response by voice using speech synthesis technology. The service provider can also provide the response in text. For example, the service provider can provide the generated response by text using text generation technology. Furthermore, the service provider can provide the response via video call. For example, the service provider can provide the generated response via video call using video call technology. By providing the generated response to the visitor, the service provider completes its interaction with the visitor. Some or all of the above processing in the service provider may be performed using, for example, a generation AI, or without a generation AI. For example, the service provider can provide the response generated by the generation AI by voice using speech synthesis technology.

[0038] The service provider can handle refusals in the case of door-to-door sales or solicitations. For example, the service provider can generate refusal statements and provide them to visitors. For example, the service provider can use generation AI to generate refusal statements for door-to-door sales or solicitations and provide them in voice using speech synthesis technology. The service provider can also use text generation technology to provide refusal statements in text. Furthermore, the service provider can use video call technology to provide refusal statements via video call. This reduces the burden on residents by allowing them to refuse door-to-door sales and solicitations. Some or all of the above processing in the service provider may be performed using, for example, generation AI, or without generation AI. For example, the service provider can provide refusal statements generated by generation AI in voice using speech synthesis technology.

[0039] The service provider can arrange for delivery by displaying a QR code. For example, in the case of delivery, the service provider can generate a QR code and display it to the visitor. For example, the service provider can use a generation AI to generate the QR code necessary for delivery and display it to the visitor. The service provider can also specify a location to display the QR code and guide the visitor there. Furthermore, the service provider can explain how to scan the QR code and guide the visitor through the delivery procedure. This reduces the effort required of residents when receiving deliveries by displaying a QR code. Some or all of the above processing in the service provider may be performed using a generation AI, for example, or without a generation AI. For example, the service provider can display a QR code generated by a generation AI to the visitor and guide them through the delivery procedure.

[0040] The service provider can prompt visitors to return when the service provider is absent. For example, the service provider can generate a message prompting visitors to return when the service provider is absent and provide it to the visitor. For example, the service provider can use generation AI to generate a message prompting visitors to return when the service provider is absent and provide it in voice using speech synthesis technology. The service provider can also use text generation technology to provide the message prompting visitors to return in text. Furthermore, the service provider can use video call technology to provide the message prompting visitors to return via video call. This facilitates communication with visitors by prompting them to return when the service provider is absent. Some or all of the above processing in the service provider may be performed using, for example, generation AI, or without generation AI. For example, the service provider can provide the message prompting visitors to return, generated by generation AI, in voice using speech synthesis technology.

[0041] The service provider can record telephone conversations. For example, the service provider can record the content of telephone conversations. For example, the service provider can use voice recording technology to record the content of telephone conversations and save it as digital data. The service provider can also use Slack transcription technology to record the content of telephone conversations as text data. Furthermore, the service provider can save the recorded data and text data to a database for later review. This allows for later review of the content of telephone conversations by recording them. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or without a generative AI. For example, the service provider can save text data generated by a generative AI to a database for later review.

[0042] The service provider can record visit images and view and share them on indoor monitors or devices. For example, the service provider can record visit images and save them as digital data. For example, the service provider can record visit images using a camera and save them as digital data. The service provider can also make the recorded visit images viewable on indoor monitors or devices. Furthermore, the service provider can share the recorded visit images with other devices. This allows residents to check the status of visitors by recording, viewing, and sharing visit images. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or without a generative AI. For example, the service provider can make the recorded data generated by the generative AI viewable on indoor monitors or devices.

[0043] The recognition unit can analyze a visitor's past visit history and optimize the recognition algorithm. For example, the recognition unit can retrieve a visitor's past visit history from a database and analyze it. For example, the recognition unit can analyze a visitor's past visit date and time and visit content to optimize the recognition algorithm. The recognition unit can also extract a visitor's characteristics based on their past visit history and optimize the recognition algorithm. Furthermore, the recognition unit can predict a visitor's behavior patterns based on their past visit history and optimize the recognition algorithm. In this way, by analyzing past visit history, the recognition algorithm is optimized and recognition accuracy is improved. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's past visit history data into a generative AI and have the generative AI perform the optimization of the recognition algorithm.

[0044] The recognition unit can improve recognition accuracy by analyzing the visitor's clothing and belongings during recognition. For example, the recognition unit can photograph the visitor's clothing color and design with a camera and analyze it. For example, the recognition unit can improve recognition accuracy by analyzing the clothing color and design using image analysis technology. The recognition unit can also photograph the luggage and small items the visitor is carrying with a camera and analyze them. For example, the recognition unit can improve recognition accuracy by analyzing the type and characteristics of the luggage and small items using image analysis technology. Furthermore, the recognition unit can also improve recognition accuracy by detecting changes in the visitor's clothing and belongings. In this way, recognition accuracy is improved by analyzing the visitor's clothing and belongings. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's image data captured by the camera into a generative AI and have the generative AI perform the analysis of the clothing and belongings.

[0045] The recognition unit can improve recognition accuracy by considering the visitor's geographical location information during recognition. For example, the recognition unit can acquire the visitor's geographical location information and reflect it in the recognition algorithm. For example, the recognition unit can acquire the visitor's location information using GPS or Wi-Fi location services and reflect it in the recognition algorithm. The recognition unit can also predict the visitor's behavior patterns based on the visitor's geographical location information and improve recognition accuracy. Furthermore, the recognition unit can extract the visitor's characteristics based on the visitor's geographical location information and improve recognition accuracy. In this way, recognition accuracy is improved by considering the visitor's geographical location information. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's geographical location information into a generative AI and have the generative AI optimize the recognition algorithm.

[0046] The recognition unit can improve recognition accuracy by analyzing the visitor's social media activity during recognition. For example, the recognition unit can improve recognition accuracy by analyzing the visitor's social media profile picture. For example, the recognition unit can improve recognition accuracy by analyzing the social media profile picture using image analysis technology. The recognition unit can also improve recognition accuracy by analyzing the content of the visitor's social media posts. For example, the recognition unit can improve recognition accuracy by analyzing the content of social media posts using text analysis technology. Furthermore, the recognition unit can improve recognition accuracy by analyzing the visitor's social media activity history. In this way, recognition accuracy is improved by analyzing the visitor's social media activity. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's social media data into a generative AI and have the generative AI perform the analysis of social media activity.

[0047] The data collection unit can improve collection accuracy by analyzing the tone of the visitor's voice and speaking style during collection. For example, the data collection unit can record and analyze the tone of the visitor's voice. For example, the data collection unit can use voice analysis technology to analyze the tone of the visitor's voice and improve collection accuracy. The data collection unit can also record and analyze the visitor's speaking style. For example, the data collection unit can analyze the speed and rhythm of the speaking style and improve collection accuracy. Furthermore, the data collection unit can detect changes in the visitor's tone of voice and speaking style to improve collection accuracy. In this way, data collection accuracy is improved by analyzing the tone of the visitor's voice and speaking style. Some or all of the above processing in the data collection unit may be performed using, for example, a generative AI, or without a generative AI. For example, the data collection unit can input the recorded visitor's voice data into a generative AI and have the generative AI perform an analysis of the tone of voice and speaking style.

[0048] The data collection unit can optimize the collected data by referring to the visitor's past visit history during collection. For example, the data collection unit can obtain and analyze the visitor's past visit history from a database. For example, the data collection unit can analyze the visitor's past visit date and time and visit content to optimize the collected data. The data collection unit can also select the collected data based on the visitor's past visit history to improve collection accuracy. Furthermore, the data collection unit can predict the visitor's behavior patterns based on the visitor's past visit history to optimize the collected data. In this way, by referring to the visitor's past visit history, the collected data is optimized and collection accuracy is improved. Some or all of the above processing in the data collection unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the data collection unit can input the visitor's past visit history data into a generative AI and have the generative AI perform the optimization of the collected data.

[0049] The response unit can provide the optimal response by referring to the visitor's past visit history when responding to a visit. For example, the response unit can retrieve and analyze the visitor's past visit history from a database. For example, the response unit can analyze the visitor's past visit date and time and visit content to provide the optimal response. The response unit can also extract the visitor's characteristics based on the visitor's past visit history and provide the optimal response. Furthermore, the response unit can predict the visitor's behavior patterns based on the visitor's past visit history and provide the optimal response. In this way, the optimal response can be provided by referring to the visitor's past visit history. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without a generative AI. For example, the response unit can input the visitor's past visit history data into a generative AI and have the generative AI perform the task of providing the optimal response.

[0050] The response unit can analyze the visitor's speaking style and tone of voice during a response to customize the response. For example, the response unit can record and analyze the visitor's speaking style. For example, it can analyze the speed and rhythm of the speaking style to customize the response. The response unit can also record and analyze the visitor's tone of voice. For example, it can analyze the pitch and volume of the voice to customize the response. Furthermore, the response unit can detect changes in the visitor's speaking style and tone of voice to customize the response. In this way, by analyzing the visitor's speaking style and tone of voice, the response can be customized, enabling a more appropriate response. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without a generative AI. For example, the response unit can input the recorded audio data of the visitor into a generative AI and have the generative AI perform the analysis of the speaking style and tone of voice.

[0051] The response unit can provide the optimal response by considering the visitor's geographical location information during the response process. For example, the response unit can acquire the visitor's geographical location information and reflect it in the response algorithm. For example, the response unit can acquire the visitor's location information using GPS or Wi-Fi location services and reflect it in the response algorithm. The response unit can also predict the visitor's behavior patterns based on the visitor's geographical location information and provide the optimal response. Furthermore, the response unit can extract the visitor's characteristics based on the visitor's geographical location information and provide the optimal response. In this way, the response unit can provide the optimal response by considering the visitor's geographical location information. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the response unit can input the visitor's geographical location information into a generative AI and have the generative AI perform the optimization of the response algorithm.

[0052] The response unit can analyze the visitor's social media activity and customize the response content at the time of response. For example, the response unit can analyze the visitor's social media profile picture and customize the response content. For example, the response unit can analyze the social media profile picture using image analysis technology and customize the response content. The response unit can also analyze the content of the visitor's social media posts and customize the response content. For example, the response unit can analyze the content of social media posts using text analysis technology and customize the response content. Furthermore, the response unit can analyze the visitor's social media activity history and customize the response content. This allows for a more appropriate response by analyzing the visitor's social media activity and customizing the response content. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without a generative AI. For example, the response unit can input the visitor's social media data into a generative AI and have the generative AI perform the analysis of social media activity.

[0053] The service delivery unit can select the optimal delivery method by referring to the visitor's past visit history at the time of delivery. For example, the service delivery unit can retrieve the visitor's past visit history from a database and analyze it. For example, the service delivery unit can analyze the visitor's past visit date and time and visit content to select the optimal delivery method. The service delivery unit can also extract the visitor's characteristics based on the visitor's past visit history and select the optimal delivery method. Furthermore, the service delivery unit can predict the visitor's behavior patterns based on the visitor's past visit history and select the optimal delivery method. In this way, the optimal delivery method can be selected by referring to the visitor's past visit history. Some or all of the above processing in the service delivery unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the service delivery unit can input the visitor's past visit history data into a generative AI and have the generative AI select the optimal delivery method.

[0054] The service provider can customize the service offered by analyzing the visitor's speaking style and tone of voice at the time of service delivery. For example, the service provider can record and analyze the visitor's speaking style. For example, it can analyze the speed and rhythm of the speaking style and customize the service offered. The service provider can also record and analyze the visitor's tone of voice. For example, it can analyze the pitch and volume of the voice and customize the service offered. Furthermore, the service provider can detect changes in the visitor's speaking style and tone of voice and customize the service offered. In this way, by analyzing the visitor's speaking style and tone of voice, the service provider can customize the service offered and provide a more appropriate service. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or not using a generative AI. For example, the service provider can input the recorded visitor's voice data into a generative AI and have the generative AI perform the analysis of the speaking style and tone of voice.

[0055] The service provider can select the optimal service delivery method by considering the visitor's geographical location information at the time of delivery. For example, the service provider can acquire the visitor's geographical location information and reflect it in the service delivery algorithm. For example, the service provider can acquire the visitor's location information using GPS or Wi-Fi location services and reflect it in the service delivery algorithm. The service provider can also predict the visitor's behavior patterns based on the visitor's geographical location information and select the optimal service delivery method. Furthermore, the service provider can extract the visitor's characteristics based on the visitor's geographical location information and select the optimal service delivery method. In this way, the optimal service delivery method can be selected by considering the visitor's geographical location information. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or without using a generative AI. For example, the service provider can input the visitor's geographical location information into a generative AI and have the generative AI perform the optimization of the service delivery algorithm.

[0056] The service provider can analyze the visitor's social media activity at the time of delivery and customize the content provided. For example, the service provider can analyze the visitor's social media profile picture and customize the content provided. For example, the service provider can analyze the social media profile picture using image analysis technology and customize the content provided. The service provider can also analyze the content of the visitor's social media posts and customize the content provided. For example, the service provider can analyze the content of social media posts using text analysis technology and customize the content provided. Furthermore, the service provider can analyze the visitor's social media activity history and customize the content provided. This allows for the customization of content and more appropriate delivery by analyzing the visitor's social media activity. Some or all of the above processing in the service provider may be performed using, for example, generative AI, or without generative AI. For example, the service provider can input the visitor's social media data into a generative AI and have the generative AI perform the analysis of social media activity.

[0057] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0058] The data collection unit can prioritize the information to be collected by referring to a visitor's past visit history. For example, if a visitor has visited frequently in the past, the data collection unit can prioritize collecting that visitor's name and purpose of visit. Similarly, if a visitor has caused trouble in the past, the data collection unit can prioritize collecting detailed information about that visitor. Furthermore, if a visitor has visited for an important purpose in the past, the data collection unit can prioritize collecting information about that purpose. This enables efficient information collection based on a visitor's past visit history.

[0059] The information service can prioritize the information to be provided by referring to a visitor's past visit history. For example, if a visitor has visited frequently in the past, the information service can provide them with information quickly. If a visitor has caused problems in the past, the information service can provide them with information cautiously. Furthermore, if a visitor has visited in the past for an important matter, the information service can provide them with detailed information. This enables efficient information provision based on the visitor's past visit history.

[0060] The recognition unit can improve the accuracy of visitor recognition by analyzing the visitor's clothing and belongings. For example, if a visitor is wearing a specific uniform, the recognition unit can identify the uniform and estimate the visitor's occupation and affiliation. Also, if a visitor is carrying luggage with a specific logo, the recognition unit can identify the logo and estimate the visitor's purpose of visit. Furthermore, if a visitor is wearing a specific accessory, the recognition unit can identify the accessory and estimate the visitor's hobbies and preferences. This enables highly accurate recognition based on the visitor's clothing and belongings.

[0061] The information gathering unit can analyze the visitor's tone of voice and speaking style to determine the priority of the information to be collected. For example, if a visitor's voice sounds tense, the unit can prioritize collecting information about that visitor. Also, if a visitor speaks in a hurry, the unit can quickly collect information about that visitor. Furthermore, if a visitor speaks calmly, the unit can carefully collect information about that visitor. This enables efficient information gathering based on the visitor's tone of voice and speaking style.

[0062] The response system can optimize its response method by referring to a visitor's past visit history. For example, if a visitor has visited frequently in the past, the response system can respond to that visitor quickly. If a visitor has caused problems in the past, the response system can respond to that visitor more carefully. Furthermore, if a visitor has visited in the past for an important matter, the response system can provide a more detailed response. This enables efficient responses based on a visitor's past visit history.

[0063] The recognition unit can improve the accuracy of visitor recognition by considering the visitor's geographical location information. For example, if a visitor is from a specific region, the recognition unit can improve the accuracy of visitor recognition by considering the characteristics of that region. Also, if a visitor is traveling along a specific route, the recognition unit can improve the accuracy of visitor recognition by considering the information of that route. Furthermore, if a visitor is visiting during a specific time period, the recognition unit can improve the accuracy of visitor recognition by considering the information of that time period. This enables highly accurate recognition based on the visitor's geographical location information.

[0064] The following briefly describes the processing flow for example form 1.

[0065] Step 1: The recognition unit recognizes the visitor. The recognition unit can recognize visitors using facial recognition technology, speech recognition technology, and motion recognition technology. For example, using facial recognition technology, the visitor's face is photographed with a camera and recognized by comparing it with a database. When using speech recognition technology, the visitor's voice is captured with a microphone and the audio data is analyzed for recognition. When using motion recognition technology, the visitor's movements are photographed with a camera and the movement patterns are analyzed for recognition. Step 2: The collection unit collects information about visitors recognized by the recognition unit. The collection unit can collect information such as the visitor's name, purpose of visit, visit time, and contact information. For example, it can ask for the visitor's name and collect it as voice data, ask for the purpose of visit and collect it as text data, ask for the purpose of visit and visit time and save it in a database, and ask for contact information and register it in the contact database. Step 3: The response unit responds to visitors based on the information collected by the collection unit. The response unit can respond according to the visitor's request, purpose of visit, and time of visit. For example, it can provide appropriate responses according to the visitor's request, necessary responses according to the purpose of visit, and prompt responses according to the time of visit. Step 4: The providing unit provides the response generated by the responding unit. The providing unit can provide the response in various formats, such as voice, text, or video call. For example, it can provide the response in voice using speech synthesis technology, in text using text generation technology, or in video call using video call technology.

[0066] (Example of form 2) The AI ​​agent reception system according to an embodiment of the present invention is a system that handles reception duties for visitors approaching a home and telephone calls, ensuring the safety and comfort of residents. This AI agent reception system supports the elderly, children, and people with disabilities by having an AI agent handle reception duties for visitors and telephone calls, eliminating inconvenience and discomfort for the user and helping to avoid trouble. For example, when a visitor approaches the door, a multimodal AI camera recognizes the visitor, and the AI ​​agent inquires about the visitor's purpose through a microphone. The generating AI analyzes the information and generates an appropriate response. For example, in the case of door-to-door sales or solicitations, the AI ​​agent will handle the refusal. In the case of deliveries, the resident can receive the package by displaying a QR code. If the resident is absent, the AI ​​agent will prompt the visitor to return. Furthermore, the AI ​​agent also handles telephone calls, recording the conversation using Slack transcription and recording functions. This allows residents to check the status of the interaction in real time. In addition, visit images are recorded and can be viewed and shared on indoor monitors or devices. This system is particularly effective for the elderly, children, people with disabilities, and women living alone, ensuring the safety and comfort of residents by supporting them in dealing with visitors. For example, it provides support tailored to individual needs, such as transcribing spoken words into text and creating example responses for the hearing impaired, and providing voice descriptions of the visitor's characteristics for the visually impaired. In this way, introducing an AI agent reception system ensures the safety and comfort of residents and can cope with the increase in teleworking, online shopping, and services like Uber in the new normal society. It also reduces the risk of crime and trouble from visitors, providing peace of mind to residents. Thus, the AI ​​agent reception system can ensure the safety and comfort of residents.

[0067] The AI ​​agent reception system according to this embodiment comprises a recognition unit, a collection unit, a response unit, and a provision unit. The recognition unit recognizes visitors. The recognition unit can recognize visitors using, for example, facial recognition technology. The recognition unit can also recognize visitors using speech recognition technology. Furthermore, the recognition unit can also recognize visitors using motion recognition technology. For example, the recognition unit uses facial recognition technology to photograph the visitor's face with a camera and recognizes it by comparing it with a database. When using speech recognition technology, the recognition unit captures the visitor's voice with a microphone and recognizes it by analyzing the audio data. When using motion recognition technology, the recognition unit photographs the visitor's movements with a camera and recognizes them by analyzing the movement patterns. The collection unit collects information about visitors recognized by the recognition unit. For example, the collection unit can collect the visitor's name and purpose of visit. Furthermore, the collection unit can also collect the visitor's purpose of visit and visit time. Furthermore, the collection unit can also collect the visitor's contact information. For example, the collection unit asks the visitor for their name and collects it as audio data. The system inquires about the visitor's purpose and collects this information as text data. It also inquires about the visitor's purpose and visit duration and stores this information in a database. Furthermore, it inquires about the visitor's contact information and registers it in the contact database. The response unit responds to the visitor based on the information collected by the collection unit. For example, the response unit can respond according to the visitor's purpose. It can also respond according to the visitor's purpose. Furthermore, it can respond according to the visitor's visit duration. For example, the response unit provides appropriate responses according to the visitor's purpose. It provides necessary responses according to the visitor's purpose. It provides prompt responses according to the visitor's visit duration. The delivery unit provides the responses generated by the response unit. For example, the delivery unit can provide responses via voice. It can also provide responses via text. Furthermore, it can provide responses via video call. For example, the delivery unit can provide responses via voice using speech synthesis technology. It can provide responses via text using text generation technology. It can provide responses via video call using video call technology. As a result, the AI ​​agent reception system according to this embodiment can consistently perform tasks from recognizing visitors to responding to them and providing answers.

[0068] The recognition unit recognizes visitors. For example, the recognition unit can recognize visitors using facial recognition technology. Specifically, it uses a camera to capture the visitor's face and inputs the image data into a facial recognition algorithm. The facial recognition algorithm compares the image with a pre-registered facial database to identify the visitor. Facial recognition technology includes methods using feature point extraction and deep learning, which enables highly accurate recognition. The recognition unit can also recognize visitors using speech recognition technology. In speech recognition technology, a microphone is used to capture the visitor's voice and inputs the audio data into a speech recognition engine. The speech recognition engine analyzes the audio data, identifies speech patterns, and converts them into text data. This allows the recognition unit to recognize the visitor's name and purpose from their voice. Furthermore, the recognition unit can also recognize visitors using motion recognition technology. In motion recognition technology, a camera is used to capture the visitor's movements and inputs the video data into a motion recognition algorithm. The motion recognition algorithm analyzes the visitor's movement patterns and identifies specific movements. For example, if a visitor waves or makes a specific gesture, the system can recognize the action and respond appropriately. This allows the recognition unit to combine various recognition technologies, including facial recognition, voice recognition, and motion recognition, to recognize the visitor from multiple perspectives.

[0069] The collection unit collects information about visitors recognized by the recognition unit. For example, the collection unit can collect the visitor's name and purpose of visit. Specifically, it asks the visitor for their name and collects the answer as voice data. The voice data is converted into text data by a speech recognition engine and stored in a database. The collection unit also asks the visitor for their purpose of visit and collects the answer as text data. The visitor's purpose of visit is collected either by converting voice to text using speech recognition technology or by the visitor entering it into a tablet or kiosk terminal. Furthermore, the collection unit can also collect the visitor's purpose of visit and visit duration. The purpose of visit is collected by asking the visitor a question and storing the answer in a database. The visit duration is collected by automatically recording the time the visitor accessed the reception system. The collection unit centrally manages this information and can link with other systems and departments as needed. For example, the collected information is stored on a cloud server and made accessible to the response and service departments. The collection unit can also collect visitor contact information. Contact information is collected by asking visitors for their phone number and email address and registering their responses in a database. This allows the data collection unit to efficiently gather detailed visitor information and improve the overall system performance.

[0070] The response department responds to visitors based on the information collected by the data collection department. For example, the response department can respond according to the visitor's request. Specifically, it can contact the appropriate person or guide the visitor through the necessary procedures, depending on the visitor's request. The response department can also respond according to the purpose of the visitor's visit. For example, if a visitor is visiting a specific department or person, it will notify that department or person of the visitor's information and request their assistance. Furthermore, the response department can respond according to the visitor's visit time. For example, if the visit time is scheduled in advance, it will prepare for the response according to that time and provide a prompt response. Based on the collected information, the response department can provide flexible responses according to the visitor's needs. For example, if a visitor has an urgent matter, it will be given priority. Also, if a visitor requests a specific service, it will provide information about that service and guide the visitor through the necessary procedures. The response department is required to provide prompt and appropriate responses in order to increase visitor satisfaction. This allows the response department to provide flexible responses according to visitor needs and improve the reliability and efficiency of the entire system.

[0071] The service provider provides the response generated by the response provider. The service provider can provide the response, for example, by voice. Specifically, it can use speech synthesis technology to provide the response generated by the response provider in voice. Speech synthesis technology converts text data into voice data and transmits it to the visitor through a speaker. The service provider can also provide the response in text. It can use text generation technology to provide the response generated by the response provider as text data. The text data can be displayed on a screen or tablet device and communicated visually to the visitor. Furthermore, the service provider can also provide the response via video call. It can use video call technology to provide the response generated by the response provider via video call. Video call technology uses a camera and microphone to communicate with the visitor in real time. This allows the service provider to provide responses to visitors in a variety of ways. For example, if a voice response is appropriate, it will be provided by voice; if a text response is appropriate, it will be provided by text; and if a video call is necessary, it will be handled by video call. This allows the service provider to provide responses in the most appropriate way according to the visitor's needs and increase visitor satisfaction. Furthermore, the service department can collect feedback from visitors and continuously improve the accuracy and effectiveness of its responses. This allows the service department to provide visitors with quick and appropriate answers, thereby improving the overall reliability and efficiency of the system.

[0072] The recognition unit can recognize visitors using a multimodal AI camera. For example, the recognition unit can recognize the visitor's face and actions using the multimodal AI camera. For example, the recognition unit can capture the visitor's face using the multimodal AI camera and recognize it using face recognition technology. The recognition unit can also capture the visitor's actions using the multimodal AI camera and recognize them using action recognition technology. Furthermore, the recognition unit can capture the visitor's voice using the multimodal AI camera and recognize it using voice recognition technology. As a result, the accuracy of visitor recognition is improved by using a multimodal AI camera. A multimodal AI camera can, for example, combine multiple sensors to simultaneously recognize multiple modals such as faces, actions, and voices. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input data acquired by the multimodal AI camera into a generative AI and have the generative AI perform visitor recognition.

[0073] The data collection unit can collect visitor requests using a sound-collecting microphone. For example, the data collection unit can use a sound-collecting microphone to capture the visitor's voice and collect their request. For example, the data collection unit can use a directional microphone to capture the visitor's voice, remove background noise, and collect their request. Alternatively, the data collection unit can use an omnidirectional microphone to capture a wide range of sounds, identify the visitor's voice, and collect their request. Furthermore, the data collection unit can use a sound-collecting microphone to analyze the visitor's voice tone and speaking style and collect their request. This allows for accurate collection of visitor requests by using a sound-collecting microphone. There are various types of sound-collecting microphones, such as high-sensitivity microphones and noise-canceling microphones, which can be selected according to the application. Some or all of the above-described processing in the data collection unit may be performed using, for example, a generative AI, or without a generative AI. For example, the data collection unit can input the audio data acquired by the sound-collecting microphone into a generative AI and have the generative AI collect visitor requests.

[0074] The response unit can generate answers to visitors' requests using a generative AI. For example, the response unit can use a generative AI to generate answers that are appropriate to the visitor's request. For example, the response unit inputs the visitor's request into the generative AI, and the generative AI generates an appropriate answer. The generative AI can use, for example, a text generation AI (e.g., LLM) to generate answers to visitors' requests. The generative AI can also use a multimodal generative AI to generate answers to visitors' requests. Furthermore, the generative AI can generate answers in formats such as voice, text, and video, depending on the visitor's request. Thus, by using the generative AI, appropriate answers to visitors' requests can be generated. The generative AI has, for example, learned from a large amount of data and possesses advanced natural language processing capabilities. The generative AI can generate the optimal answer according to the visitor's request. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the response unit can input the visitor's request into the generative AI and provide the answer generated by the generative AI.

[0075] The service provider can provide the generated response to the visitor. The service provider can provide the response, for example, by voice. For example, the service provider can provide the generated response by voice using speech synthesis technology. The service provider can also provide the response in text. For example, the service provider can provide the generated response by text using text generation technology. Furthermore, the service provider can provide the response via video call. For example, the service provider can provide the generated response via video call using video call technology. By providing the generated response to the visitor, the service provider completes its interaction with the visitor. Some or all of the above processing in the service provider may be performed using, for example, a generation AI, or without a generation AI. For example, the service provider can provide the response generated by the generation AI by voice using speech synthesis technology.

[0076] The service provider can handle refusals in the case of door-to-door sales or solicitations. For example, the service provider can generate refusal statements and provide them to visitors. For example, the service provider can use generation AI to generate refusal statements for door-to-door sales or solicitations and provide them in voice using speech synthesis technology. The service provider can also use text generation technology to provide refusal statements in text. Furthermore, the service provider can use video call technology to provide refusal statements via video call. This reduces the burden on residents by allowing them to refuse door-to-door sales and solicitations. Some or all of the above processing in the service provider may be performed using, for example, generation AI, or without generation AI. For example, the service provider can provide refusal statements generated by generation AI in voice using speech synthesis technology.

[0077] The service provider can arrange for delivery by displaying a QR code. For example, in the case of delivery, the service provider can generate a QR code and display it to the visitor. For example, the service provider can use a generation AI to generate the QR code necessary for delivery and display it to the visitor. The service provider can also specify a location to display the QR code and guide the visitor there. Furthermore, the service provider can explain how to scan the QR code and guide the visitor through the delivery procedure. This reduces the effort required of residents when receiving deliveries by displaying a QR code. Some or all of the above processing in the service provider may be performed using a generation AI, for example, or without a generation AI. For example, the service provider can display a QR code generated by a generation AI to the visitor and guide them through the delivery procedure.

[0078] The service provider can prompt visitors to return when the service provider is absent. For example, the service provider can generate a message prompting visitors to return when the service provider is absent and provide it to the visitor. For example, the service provider can use generation AI to generate a message prompting visitors to return when the service provider is absent and provide it in voice using speech synthesis technology. The service provider can also use text generation technology to provide the message prompting visitors to return in text. Furthermore, the service provider can use video call technology to provide the message prompting visitors to return via video call. This facilitates communication with visitors by prompting them to return when the service provider is absent. Some or all of the above processing in the service provider may be performed using, for example, generation AI, or without generation AI. For example, the service provider can provide the message prompting visitors to return, generated by generation AI, in voice using speech synthesis technology.

[0079] The service provider can record telephone conversations. For example, the service provider can record the content of telephone conversations. For example, the service provider can use voice recording technology to record the content of telephone conversations and save it as digital data. The service provider can also use Slack transcription technology to record the content of telephone conversations as text data. Furthermore, the service provider can save the recorded data and text data to a database for later review. This allows for later review of the content of telephone conversations by recording them. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or without a generative AI. For example, the service provider can save text data generated by a generative AI to a database for later review.

[0080] The service provider can record visit images and view and share them on indoor monitors or devices. For example, the service provider can record visit images and save them as digital data. For example, the service provider can record visit images using a camera and save them as digital data. The service provider can also make the recorded visit images viewable on indoor monitors or devices. Furthermore, the service provider can share the recorded visit images with other devices. This allows residents to check the status of visitors by recording, viewing, and sharing visit images. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or without a generative AI. For example, the service provider can make the recorded data generated by the generative AI viewable on indoor monitors or devices.

[0081] The recognition unit can estimate the visitor's emotions and adjust the recognition accuracy based on the estimated emotions. For example, the recognition unit can capture the visitor's facial expression with a camera and estimate the emotion using an emotion estimation algorithm. For example, the recognition unit can estimate the visitor's emotions using facial expression recognition technology. The recognition unit can also capture the visitor's voice and estimate the emotion using speech analysis technology. Furthermore, the recognition unit can capture the visitor's movements with a camera and estimate the emotion using motion analysis technology. By adjusting the recognition accuracy based on the visitor's emotions, the accuracy of recognition is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the recognition unit may be performed using a generative AI, for example, or without a generative AI. For example, the recognition unit can input image data of visitors captured by a camera into a generating AI, which can then perform the estimation of the visitors' emotions.

[0082] The recognition unit can analyze a visitor's past visit history and optimize the recognition algorithm. For example, the recognition unit can retrieve a visitor's past visit history from a database and analyze it. For example, the recognition unit can analyze a visitor's past visit date and time and visit content to optimize the recognition algorithm. The recognition unit can also extract a visitor's characteristics based on their past visit history and optimize the recognition algorithm. Furthermore, the recognition unit can predict a visitor's behavior patterns based on their past visit history and optimize the recognition algorithm. In this way, by analyzing past visit history, the recognition algorithm is optimized and recognition accuracy is improved. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's past visit history data into a generative AI and have the generative AI perform the optimization of the recognition algorithm.

[0083] The recognition unit can improve recognition accuracy by analyzing the visitor's clothing and belongings during recognition. For example, the recognition unit can photograph the visitor's clothing color and design with a camera and analyze it. For example, the recognition unit can improve recognition accuracy by analyzing the clothing color and design using image analysis technology. The recognition unit can also photograph the luggage and small items the visitor is carrying with a camera and analyze them. For example, the recognition unit can improve recognition accuracy by analyzing the type and characteristics of the luggage and small items using image analysis technology. Furthermore, the recognition unit can also improve recognition accuracy by detecting changes in the visitor's clothing and belongings. In this way, recognition accuracy is improved by analyzing the visitor's clothing and belongings. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's image data captured by the camera into a generative AI and have the generative AI perform the analysis of the clothing and belongings.

[0084] The recognition unit can estimate the visitor's emotions and adjust the display method of the recognition results based on the estimated visitor's emotions. For example, the recognition unit can capture the visitor's facial expression with a camera and estimate the emotion using an emotion estimation algorithm. For example, the recognition unit can estimate the visitor's emotions using facial expression recognition technology. The recognition unit can also capture the visitor's voice and estimate the emotion using speech analysis technology. Furthermore, the recognition unit can capture the visitor's movements with a camera and estimate the emotion using motion analysis technology. This allows residents to respond quickly by adjusting the display method of the recognition results based on the visitor's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the recognition unit may be performed using a generative AI, for example, or without a generative AI. For example, the recognition unit can input image data of visitors captured by a camera into a generating AI, which can then perform the estimation of the visitors' emotions.

[0085] The recognition unit can improve recognition accuracy by considering the visitor's geographical location information during recognition. For example, the recognition unit can acquire the visitor's geographical location information and reflect it in the recognition algorithm. For example, the recognition unit can acquire the visitor's location information using GPS or Wi-Fi location services and reflect it in the recognition algorithm. The recognition unit can also predict the visitor's behavior patterns based on the visitor's geographical location information and improve recognition accuracy. Furthermore, the recognition unit can extract the visitor's characteristics based on the visitor's geographical location information and improve recognition accuracy. In this way, recognition accuracy is improved by considering the visitor's geographical location information. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's geographical location information into a generative AI and have the generative AI optimize the recognition algorithm.

[0086] The recognition unit can improve recognition accuracy by analyzing the visitor's social media activity during recognition. For example, the recognition unit can improve recognition accuracy by analyzing the visitor's social media profile picture. For example, the recognition unit can improve recognition accuracy by analyzing the social media profile picture using image analysis technology. The recognition unit can also improve recognition accuracy by analyzing the content of the visitor's social media posts. For example, the recognition unit can improve recognition accuracy by analyzing the content of social media posts using text analysis technology. Furthermore, the recognition unit can improve recognition accuracy by analyzing the visitor's social media activity history. In this way, recognition accuracy is improved by analyzing the visitor's social media activity. Some or all of the above processing in the recognition unit may be performed using, for example, a generative AI, or without a generative AI. For example, the recognition unit can input the visitor's social media data into a generative AI and have the generative AI perform the analysis of social media activity.

[0087] The data collection unit can estimate the emotions of visitors and adjust the collection method based on the estimated emotions. For example, the data collection unit can capture the visitor's facial expression with a camera and estimate the emotion using an emotion estimation algorithm. For example, the data collection unit can estimate the visitor's emotions using facial recognition technology. The data collection unit can also capture the visitor's voice and estimate the emotion using speech analysis technology. Furthermore, the data collection unit can capture the visitor's movements with a camera and estimate the emotion using motion analysis technology. By adjusting the collection method based on the visitor's emotions, the collection accuracy is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the data collection unit may be performed using a generative AI, for example, or without a generative AI. For example, the data collection unit can input image data of visitors captured by a camera into a generating AI, which can then perform the estimation of the visitors' emotions.

[0088] The data collection unit can improve collection accuracy by analyzing the tone of the visitor's voice and speaking style during collection. For example, the data collection unit can record and analyze the tone of the visitor's voice. For example, the data collection unit can use voice analysis technology to analyze the tone of the visitor's voice and improve collection accuracy. The data collection unit can also record and analyze the visitor's speaking style. For example, the data collection unit can analyze the speed and rhythm of the speaking style and improve collection accuracy. Furthermore, the data collection unit can detect changes in the visitor's tone of voice and speaking style to improve collection accuracy. In this way, data collection accuracy is improved by analyzing the tone of the visitor's voice and speaking style. Some or all of the above processing in the data collection unit may be performed using, for example, a generative AI, or without a generative AI. For example, the data collection unit can input the recorded visitor's voice data into a generative AI and have the generative AI perform an analysis of the tone of voice and speaking style.

[0089] The data collection unit can estimate the emotions of visitors and determine the priority of the collected content based on the estimated emotions. For example, the data collection unit can capture the visitor's facial expression with a camera and estimate the emotion using an emotion estimation algorithm. For example, the data collection unit can estimate the visitor's emotion using facial recognition technology. The data collection unit can also capture the visitor's voice and estimate the emotion using speech analysis technology. Furthermore, the data collection unit can capture the visitor's movements with a camera and estimate the emotion using motion analysis technology. By determining the priority of the collected content based on the visitor's emotions, important information can be collected preferentially. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the data collection unit may be performed using a generative AI, for example, or without a generative AI. For example, the data collection unit can input image data of visitors captured by a camera into a generating AI, which can then perform the estimation of the visitors' emotions.

[0090] The data collection unit can optimize the collected data by referring to the visitor's past visit history during collection. For example, the data collection unit can obtain and analyze the visitor's past visit history from a database. For example, the data collection unit can analyze the visitor's past visit date and time and visit content to optimize the collected data. The data collection unit can also select the collected data based on the visitor's past visit history to improve collection accuracy. Furthermore, the data collection unit can predict the visitor's behavior patterns based on the visitor's past visit history to optimize the collected data. In this way, by referring to the visitor's past visit history, the collected data is optimized and collection accuracy is improved. Some or all of the above processing in the data collection unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the data collection unit can input the visitor's past visit history data into a generative AI and have the generative AI perform the optimization of the collected data.

[0091] The response unit can estimate the visitor's emotions and adjust its response method based on the estimated emotions. For example, the response unit can capture the visitor's facial expression with a camera and estimate the emotion using an emotion estimation algorithm. For example, the response unit can estimate the visitor's emotions using facial recognition technology. The response unit can also capture the visitor's voice and estimate the emotion using speech analysis technology. Furthermore, the response unit can capture the visitor's movements with a camera and estimate the emotion using motion analysis technology. By adjusting the response method based on the visitor's emotions, the accuracy of the response to the visitor is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the response unit may be performed using a generative AI, for example, or without a generative AI. For example, the response unit can input image data of visitors captured by a camera into a generating AI, and have the generating AI perform an estimation of the visitors' emotions.

[0092] The response unit can provide the optimal response by referring to the visitor's past visit history when responding to a visit. For example, the response unit can retrieve and analyze the visitor's past visit history from a database. For example, the response unit can analyze the visitor's past visit date and time and visit content to provide the optimal response. The response unit can also extract the visitor's characteristics based on the visitor's past visit history and provide the optimal response. Furthermore, the response unit can predict the visitor's behavior patterns based on the visitor's past visit history and provide the optimal response. In this way, the optimal response can be provided by referring to the visitor's past visit history. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without a generative AI. For example, the response unit can input the visitor's past visit history data into a generative AI and have the generative AI perform the task of providing the optimal response.

[0093] The response unit can analyze the visitor's speaking style and tone of voice during a response to customize the response. For example, the response unit can record and analyze the visitor's speaking style. For example, it can analyze the speed and rhythm of the speaking style to customize the response. The response unit can also record and analyze the visitor's tone of voice. For example, it can analyze the pitch and volume of the voice to customize the response. Furthermore, the response unit can detect changes in the visitor's speaking style and tone of voice to customize the response. In this way, by analyzing the visitor's speaking style and tone of voice, the response can be customized, enabling a more appropriate response. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without a generative AI. For example, the response unit can input the recorded audio data of the visitor into a generative AI and have the generative AI perform the analysis of the speaking style and tone of voice.

[0094] The response unit can estimate the visitor's emotions and determine the priority of the response based on the estimated emotions. For example, the response unit can capture the visitor's facial expression with a camera and estimate the emotion using an emotion estimation algorithm. For example, the response unit can estimate the visitor's emotions using facial recognition technology. The response unit can also capture the visitor's voice and estimate the emotion using speech analysis technology. Furthermore, the response unit can capture the visitor's movements with a camera and estimate the emotion using motion analysis technology. By determining the priority of the response based on the visitor's emotions, important information can be prioritized. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the response unit may be performed using a generative AI, for example, or without a generative AI. For example, the response unit can input image data of visitors captured by a camera into a generating AI, and have the generating AI perform an estimation of the visitors' emotions.

[0095] The response unit can provide the optimal response by considering the visitor's geographical location information during the response process. For example, the response unit can acquire the visitor's geographical location information and reflect it in the response algorithm. For example, the response unit can acquire the visitor's location information using GPS or Wi-Fi location services and reflect it in the response algorithm. The response unit can also predict the visitor's behavior patterns based on the visitor's geographical location information and provide the optimal response. Furthermore, the response unit can extract the visitor's characteristics based on the visitor's geographical location information and provide the optimal response. In this way, the response unit can provide the optimal response by considering the visitor's geographical location information. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the response unit can input the visitor's geographical location information into a generative AI and have the generative AI perform the optimization of the response algorithm.

[0096] The response unit can analyze the visitor's social media activity and customize the response content at the time of response. For example, the response unit can analyze the visitor's social media profile picture and customize the response content. For example, the response unit can analyze the social media profile picture using image analysis technology and customize the response content. The response unit can also analyze the content of the visitor's social media posts and customize the response content. For example, the response unit can analyze the content of social media posts using text analysis technology and customize the response content. Furthermore, the response unit can analyze the visitor's social media activity history and customize the response content. This allows for a more appropriate response by analyzing the visitor's social media activity and customizing the response content. Some or all of the above processing in the response unit may be performed using, for example, a generative AI, or without a generative AI. For example, the response unit can input the visitor's social media data into a generative AI and have the generative AI perform the analysis of social media activity.

[0097] The service provider can estimate the visitor's emotions and adjust the service delivery method based on the estimated emotions. For example, the service provider can capture the visitor's facial expressions with a camera and estimate their emotions using an emotion estimation algorithm. For example, the service provider can estimate the visitor's emotions using facial recognition technology. The service provider can also capture the visitor's voice and estimate their emotions using speech analysis technology. Furthermore, the service provider can capture the visitor's movements with a camera and estimate their emotions using motion analysis technology. By adjusting the service delivery method based on the visitor's emotions, the accuracy of the service delivered is improved. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the service provider may be performed using a generative AI, for example, or without a generative AI. For example, the service provider can input image data of visitors captured by a camera into a generating AI, and have the AI ​​perform an estimation of the visitors' emotions.

[0098] The service delivery unit can select the optimal delivery method by referring to the visitor's past visit history at the time of delivery. For example, the service delivery unit can retrieve the visitor's past visit history from a database and analyze it. For example, the service delivery unit can analyze the visitor's past visit date and time and visit content to select the optimal delivery method. The service delivery unit can also extract the visitor's characteristics based on the visitor's past visit history and select the optimal delivery method. Furthermore, the service delivery unit can predict the visitor's behavior patterns based on the visitor's past visit history and select the optimal delivery method. In this way, the optimal delivery method can be selected by referring to the visitor's past visit history. Some or all of the above processing in the service delivery unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the service delivery unit can input the visitor's past visit history data into a generative AI and have the generative AI select the optimal delivery method.

[0099] The service provider can customize the service offered by analyzing the visitor's speaking style and tone of voice at the time of service delivery. For example, the service provider can record and analyze the visitor's speaking style. For example, it can analyze the speed and rhythm of the speaking style and customize the service offered. The service provider can also record and analyze the visitor's tone of voice. For example, it can analyze the pitch and volume of the voice and customize the service offered. Furthermore, the service provider can detect changes in the visitor's speaking style and tone of voice and customize the service offered. In this way, by analyzing the visitor's speaking style and tone of voice, the service provider can customize the service offered and provide a more appropriate service. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or not using a generative AI. For example, the service provider can input the recorded visitor's voice data into a generative AI and have the generative AI perform the analysis of the speaking style and tone of voice.

[0100] The service provider can estimate the visitor's emotions and determine the priority of the content offered based on the estimated emotions. For example, the service provider can capture the visitor's facial expressions with a camera and estimate their emotions using an emotion estimation algorithm. For example, the service provider can estimate the visitor's emotions using facial recognition technology. The service provider can also capture the visitor's voice and estimate their emotions using speech analysis technology. Furthermore, the service provider can capture the visitor's movements with a camera and estimate their emotions using motion analysis technology. By determining the priority of the content offered based on the visitor's emotions, important information can be provided preferentially. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. Generative AI is, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the service provider may be performed using a generative AI, for example, or without a generative AI. For example, the service provider can input image data of visitors captured by a camera into a generating AI, and have the AI ​​perform an estimation of the visitors' emotions.

[0101] The service provider can select the optimal service delivery method by considering the visitor's geographical location information at the time of delivery. For example, the service provider can acquire the visitor's geographical location information and reflect it in the service delivery algorithm. For example, the service provider can acquire the visitor's location information using GPS or Wi-Fi location services and reflect it in the service delivery algorithm. The service provider can also predict the visitor's behavior patterns based on the visitor's geographical location information and select the optimal service delivery method. Furthermore, the service provider can extract the visitor's characteristics based on the visitor's geographical location information and select the optimal service delivery method. In this way, the optimal service delivery method can be selected by considering the visitor's geographical location information. Some or all of the above processing in the service provider may be performed using, for example, a generative AI, or without using a generative AI. For example, the service provider can input the visitor's geographical location information into a generative AI and have the generative AI perform the optimization of the service delivery algorithm.

[0102] The service provider can analyze the visitor's social media activity at the time of delivery and customize the content provided. For example, the service provider can analyze the visitor's social media profile picture and customize the content provided. For example, the service provider can analyze the social media profile picture using image analysis technology and customize the content provided. The service provider can also analyze the content of the visitor's social media posts and customize the content provided. For example, the service provider can analyze the content of social media posts using text analysis technology and customize the content provided. Furthermore, the service provider can analyze the visitor's social media activity history and customize the content provided. This allows for the customization of content and more appropriate delivery by analyzing the visitor's social media activity. Some or all of the above processing in the service provider may be performed using, for example, generative AI, or without generative AI. For example, the service provider can input the visitor's social media data into a generative AI and have the generative AI perform the analysis of social media activity.

[0103] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0104] The recognition unit can estimate the visitor's emotions and adjust its recognition accuracy based on those emotions. For example, if a visitor looks anxious, the recognition unit can take that emotion into consideration and adjust its response to be more courteous. Similarly, if a visitor is angry, the recognition unit can take that emotion into consideration and adjust its response to be more swift. Furthermore, if a visitor is happy, the recognition unit can take that emotion into consideration and adjust its response to be more friendly. This enables flexible responses that are tailored to the visitor's emotions.

[0105] The data collection unit can prioritize the information to be collected by referring to a visitor's past visit history. For example, if a visitor has visited frequently in the past, the data collection unit can prioritize collecting that visitor's name and purpose of visit. Similarly, if a visitor has caused trouble in the past, the data collection unit can prioritize collecting detailed information about that visitor. Furthermore, if a visitor has visited for an important purpose in the past, the data collection unit can prioritize collecting information about that purpose. This enables efficient information collection based on a visitor's past visit history.

[0106] The response unit can estimate the visitor's emotions and adjust its response based on those estimates. For example, if a visitor looks anxious, the response unit can take that emotion into consideration and respond in a reassuring manner. If a visitor is angry, the response unit can take that emotion into consideration and respond quickly and calmly. Furthermore, if a visitor is happy, the response unit can take that emotion into consideration and respond in a friendly manner. This allows for flexible responses that are tailored to the visitor's emotions.

[0107] The information service can prioritize the information to be provided by referring to a visitor's past visit history. For example, if a visitor has visited frequently in the past, the information service can provide them with information quickly. If a visitor has caused problems in the past, the information service can provide them with information cautiously. Furthermore, if a visitor has visited in the past for an important matter, the information service can provide them with detailed information. This enables efficient information provision based on the visitor's past visit history.

[0108] The recognition unit can improve the accuracy of visitor recognition by analyzing the visitor's clothing and belongings. For example, if a visitor is wearing a specific uniform, the recognition unit can identify the uniform and estimate the visitor's occupation and affiliation. Also, if a visitor is carrying luggage with a specific logo, the recognition unit can identify the logo and estimate the visitor's purpose of visit. Furthermore, if a visitor is wearing a specific accessory, the recognition unit can identify the accessory and estimate the visitor's hobbies and preferences. This enables highly accurate recognition based on the visitor's clothing and belongings.

[0109] The information gathering unit can analyze the visitor's tone of voice and speaking style to determine the priority of the information to be collected. For example, if a visitor's voice sounds tense, the unit can prioritize collecting information about that visitor. Also, if a visitor speaks in a hurry, the unit can quickly collect information about that visitor. Furthermore, if a visitor speaks calmly, the unit can carefully collect information about that visitor. This enables efficient information gathering based on the visitor's tone of voice and speaking style.

[0110] The response system can optimize its response method by referring to a visitor's past visit history. For example, if a visitor has visited frequently in the past, the response system can respond to that visitor quickly. If a visitor has caused problems in the past, the response system can respond to that visitor more carefully. Furthermore, if a visitor has visited in the past for an important matter, the response system can provide a more detailed response. This enables efficient responses based on a visitor's past visit history.

[0111] The information delivery system can estimate the visitor's emotions and prioritize the information to be provided based on those estimates. For example, if a visitor looks anxious, the system can take that emotion into consideration and prioritize providing reassuring information. If a visitor is angry, the system can take that emotion into consideration and provide information quickly. Furthermore, if a visitor is happy, the system can take that emotion into consideration and provide friendly information. This enables flexible information delivery that responds to the visitor's emotions.

[0112] The recognition unit can improve the accuracy of visitor recognition by considering the visitor's geographical location information. For example, if a visitor is from a specific region, the recognition unit can improve the accuracy of visitor recognition by considering the characteristics of that region. Also, if a visitor is traveling along a specific route, the recognition unit can improve the accuracy of visitor recognition by considering the information of that route. Furthermore, if a visitor is visiting during a specific time period, the recognition unit can improve the accuracy of visitor recognition by considering the information of that time period. This enables highly accurate recognition based on the visitor's geographical location information.

[0113] The information gathering unit can estimate the visitor's emotions and prioritize the information to collect based on those estimates. For example, if a visitor looks anxious, the unit can take that emotion into consideration and prioritize collecting reassuring information. If a visitor is angry, the unit can take that emotion into consideration and quickly collect information. Furthermore, if a visitor is happy, the unit can take that emotion into consideration and collect friendly information. This enables flexible information gathering that responds to the visitor's emotions.

[0114] The following briefly describes the processing flow for example form 2.

[0115] Step 1: The recognition unit recognizes the visitor. The recognition unit can recognize visitors using facial recognition technology, speech recognition technology, and motion recognition technology. For example, using facial recognition technology, the visitor's face is photographed with a camera and recognized by comparing it with a database. When using speech recognition technology, the visitor's voice is captured with a microphone and the audio data is analyzed for recognition. When using motion recognition technology, the visitor's movements are photographed with a camera and the movement patterns are analyzed for recognition. Step 2: The collection unit collects information about visitors recognized by the recognition unit. The collection unit can collect information such as the visitor's name, purpose of visit, visit time, and contact information. For example, it can ask for the visitor's name and collect it as voice data, ask for the purpose of visit and collect it as text data, ask for the purpose of visit and visit time and save it in a database, and ask for contact information and register it in the contact database. Step 3: The response unit responds to visitors based on the information collected by the collection unit. The response unit can respond according to the visitor's request, purpose of visit, and time of visit. For example, it can provide appropriate responses according to the visitor's request, necessary responses according to the purpose of visit, and prompt responses according to the time of visit. Step 4: The providing unit provides the response generated by the responding unit. The providing unit can provide the response in various formats, such as voice, text, or video call. For example, it can provide the response in voice using speech synthesis technology, in text using text generation technology, or in video call using video call technology.

[0116] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0117] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0118] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0119] Each of the multiple elements described above, including the recognition unit, collection unit, response unit, and provision unit, is implemented, for example, in at least one of the smart device 14 and the data processing unit 12. For example, the recognition unit recognizes visitors using the camera 42 and microphone 38B of the smart device 14 and performs facial recognition, voice recognition, or motion recognition technology using the control unit 46A. The collection unit collects visitor information using the control unit 46A of the smart device 14. The response unit responds to visitors based on the information collected by the identification processing unit 290 of the data processing unit 12. The provision unit provides responses via voice, text, or video call using the control unit 46A of the smart device 14. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0120] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0121] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0122] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0123] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0124] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0125] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0126] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0127] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0128] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0129] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0130] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0131] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0132] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0133] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0134] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0135] Each of the multiple elements described above, including the recognition unit, collection unit, response unit, and provision unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the recognition unit recognizes visitors using the camera 42 and microphone 238 of the smart glasses 214 and performs face recognition, voice recognition, or motion recognition technology using the control unit 46A. The collection unit collects visitor information using the control unit 46A of the smart glasses 214. The response unit responds to visitors based on the information collected by the identification processing unit 290 of the data processing unit 12. The provision unit provides responses via voice, text, or video call using the control unit 46A of the smart glasses 214. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0136] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0137] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0138] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0139] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0140] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0141] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0142] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0143] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0144] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0145] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0146] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0147] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0148] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0149] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0150] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0151] Each of the multiple elements described above, including the recognition unit, collection unit, response unit, and provision unit, is implemented, for example, in at least one of the headset terminal 314 and the data processing unit 12. For example, the recognition unit recognizes visitors using the camera 42 and microphone 238 of the headset terminal 314 and performs facial recognition, voice recognition, or motion recognition technology using the control unit 46A. The collection unit collects visitor information using the control unit 46A of the headset terminal 314. The response unit responds to visitors based on the information collected by the identification processing unit 290 of the data processing unit 12. The provision unit provides responses via voice, text, or video call using the control unit 46A of the headset terminal 314. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0152] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0153] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0154] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0155] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0156] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0157] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0158] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0159] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0160] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0161] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0162] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0163] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0164] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0165] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0166] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0167] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0168] Each of the multiple elements described above, including the recognition unit, collection unit, response unit, and provision unit, is implemented, for example, in at least one of the robot 414 and the data processing unit 12. For example, the recognition unit recognizes visitors using the camera 42 and microphone 238 of the robot 414 and performs face recognition, voice recognition, or motion recognition technology using the control unit 46A. The collection unit collects visitor information using the control unit 46A of the robot 414. The response unit responds to visitors based on the information collected by the identification processing unit 290 of the data processing unit 12. The provision unit provides responses via voice, text, or video call using the control unit 46A of the robot 414. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0169] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0170] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0171] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0172] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0173] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0174] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0175] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0176] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0177] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0178] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0179] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0180] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0181] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0182] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0183] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0184] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0185] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0186] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0187] (Note 1) A recognition unit that recognizes visitors, A collection unit that collects information on visitors recognized by the recognition unit, Based on the information collected by the aforementioned collection unit, a response unit is established to deal with visitors, A providing unit that provides the answer generated by the aforementioned corresponding unit, Equipped with A system characterized by the following features. (Note 2) The aforementioned recognition unit, Recognizing visitors using multimodal AI cameras The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned collection unit is Use a sound-collecting microphone to gather information about the visitor's purpose. The system described in Appendix 1, characterized by the features described herein. (Note 4) The corresponding part is, Generating responses to visitors' requests using generative AI. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned supply unit is, Provide the generated response to the visitor. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned supply unit is, In the case of door-to-door sales or solicitations, you should refuse. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned supply unit is, For home delivery, present the QR code to receive your order. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned supply unit is, If you are not home, please ask the visitor to return later. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned supply unit is, Record phone conversations. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned supply unit is, Record visit images and view / share them on indoor monitors or devices. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned recognition unit, The system estimates the visitor's emotions and adjusts the recognition accuracy based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned recognition unit, Analyze visitors' past visit history to optimize the recognition algorithm. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned recognition unit, During recognition, the system analyzes the visitor's clothing and belongings to improve recognition accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned recognition unit, The system estimates the visitor's emotions and adjusts the display method of the recognition results based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned recognition unit, During recognition, the system improves recognition accuracy by taking into account the visitor's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned recognition unit, During recognition, the system analyzes the visitor's social media activity to improve recognition accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned collection unit is We estimate the emotions of visitors and adjust the collection method based on the estimated emotions of the visitors. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned collection unit is During data collection, we analyze the tone of voice and speaking style of visitors to improve collection accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned collection unit is The system estimates the visitor's emotions and prioritizes the information to be collected based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned collection unit is During data collection, the system optimizes the collected data by referencing the visitor's past visit history. The system described in Appendix 1, characterized by the features described herein. (Note 21) The corresponding part is, The system estimates the visitor's emotions and adjusts its response based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 22) The corresponding part is, When responding to a visitor, we refer to their past visit history to provide the most appropriate response. The system described in Appendix 1, characterized by the features described herein. (Note 23) The corresponding part is, During interaction, the system analyzes the visitor's speaking style and tone of voice to customize the response accordingly. The system described in Appendix 1, characterized by the features described herein. (Note 24) The corresponding part is, The system estimates the visitor's emotions and prioritizes the appropriate response based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 25) The corresponding part is, When responding to a visitor, we provide the most appropriate response by taking into account the visitor's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 26) The corresponding part is, When responding to visitors, we analyze their social media activity to customize our response accordingly. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned supply unit is, The system estimates the visitor's emotions and adjusts the service delivery method based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned supply unit is, When providing the service, the optimal delivery method is selected by referring to the visitor's past visit history. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned supply unit is, During the service delivery, the system analyzes the visitor's speaking style and tone of voice to customize the service provided. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned supply unit is, The system estimates the visitor's emotions and prioritizes the offerings based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned supply unit is, When providing the service, the optimal delivery method will be selected considering the visitor's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned supply unit is, When providing the service, we analyze the visitor's social media activity to customize the content. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0188] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A recognition unit that recognizes visitors, A collection unit that collects information on visitors recognized by the recognition unit, Based on the information collected by the aforementioned collection unit, a response unit is established to deal with visitors, A providing unit that provides the answer generated by the aforementioned corresponding unit, Equipped with A system characterized by the following features.

2. The aforementioned recognition unit, Recognizing visitors using a multimodal AI camera. The system according to feature 1.

3. The aforementioned collection unit is Use a sound-collecting microphone to gather information about the visitor's purpose. The system according to feature 1.

4. The corresponding part is, Use generative AI to generate responses to visitors' requests. The system according to feature 1.

5. The aforementioned supply unit is, Provide the generated response to the visitor. The system according to feature 1.

6. The aforementioned supply unit is, In the case of door-to-door sales or solicitations, you should refuse. The system according to feature 1.

7. The aforementioned supply unit is, If you are not home, please ask the visitor to return later. The system according to feature 1.

8. The aforementioned supply unit is, Record phone conversations. The system according to feature 1.

9. The aforementioned supply unit is, Record visit images and view / share them on indoor monitors or devices. The system according to feature 1.

Images