system

Abstract

The system according to this embodiment provides support for starting operations in a short amount of time, with the aim of understanding work procedures and preventing errors. [Solution] The system according to the embodiment comprises a reception unit, a guidance unit, a display unit, and an alert unit. The reception unit receives user input. The guidance unit provides voice guidance based on the information received by the reception unit. The display unit displays the work procedure in augmented reality based on the voice guidance provided by the guidance unit. The alert unit provides error prevention and prevention alerts based on the work procedure displayed by the display unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, the support for starting operations in a short time is not sufficient, and there are problems in understanding operation procedures and preventing errors.

[0005] The system according to the embodiment provides support for starting operations in a short time, aiming at understanding operation procedures and preventing errors.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a reception unit, a guidance unit, a display unit, and an alert unit. The reception unit receives user input. The guidance unit provides voice guidance based on the information received by the reception unit. The display unit displays the work procedure using augmented reality (AR) based on the voice guidance provided by the guidance unit. The alert unit provides error prevention and prevention alerts based on the work procedure displayed by the display unit. [Effects of the Invention]

[0007] The system according to this embodiment provides support for starting operations in a short amount of time, and can facilitate understanding of work procedures and prevent errors. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The AI ​​agent system according to an embodiment of the present invention is a system for starting work quickly in today's world where the demand for part-time jobs and side jobs is increasing. This system explains the basic procedures of a task to the user through voice guidance, visually displays the procedures using AR display, and provides error prevention and prevention alerts. For example, when a user starts work, the AI ​​agent system explains the basic procedures of the task through voice guidance. For example, in the case of customer service, the AI ​​agent system provides voice guidance such as, "I'll teach you how to serve customers in this section. Our best-selling products are over here. If you have any questions, feel free to ask anytime." Next, the AI ​​agent system visually displays the procedures using AR display. For example, the user can check the procedures using a smartphone or AR glasses. This makes it easier for the user to visually understand the procedures. Furthermore, the AI ​​agent system provides error prevention and prevention alerts to support the user in accurately performing the task. For example, if the user makes a mistake in the procedure, the AI ​​agent system provides an alert such as, "That procedure is incorrect. The correct procedure is here." This mechanism allows work to be started quickly, and enables efficient work even when it is difficult to secure training time or when it takes time to understand the work manual. Potential target users include company employees, students, housewives / househusbands, and freelancers seeking side jobs. This would allow the AI ​​agent system to support users in starting and accurately completing tasks in a short amount of time.

[0029] The AI ​​agent system according to this embodiment comprises a reception unit, a guidance unit, a display unit, and an alert unit. The reception unit receives user input. User input includes, but is not limited to, voice input, touch input, and keyboard input. The reception unit receives user instructions using, for example, voice input. The reception unit can also receive user instructions using touch input. The reception unit can also receive user instructions using keyboard input. For example, the reception unit analyzes the user's voice input using speech recognition technology and receives instructions. Touch input is a method in which the user inputs instructions by operating a touch panel, allowing for intuitive operation. Keyboard input is a method in which the user inputs instructions using a keyboard, allowing for accurate input. The guidance unit provides voice guidance based on the information received by the reception unit. Voice guidance is provided based on, for example, the type of voice, the content of the guidance, the timing of provision, etc., but is not limited to these examples. For example, the guidance unit explains the basic procedures of the work to the user by voice. The guidance unit can also provide information necessary for the user to start the work by voice. Furthermore, the guidance unit can provide voice guidance to correct the procedure if the user makes a mistake in the work procedure. For example, the guidance unit can provide guidance in a natural voice using speech synthesis technology. The voice guidance is explained in easy-to-understand language so that the user can easily understand the work. The content of the guidance is appropriately adjusted according to the type and situation of the work. The display unit displays the work procedure in AR based on the voice guidance provided by the guidance unit. The AR display is performed using, for example, a smartphone or AR glasses, but is not limited to these examples. For example, the display unit displays the work procedure on the screen of a smartphone. The display unit can also visually show the work procedure using AR glasses. The display unit can also show each step of the work procedure in detail. For example, the display unit can overlay the work procedure onto the real-world scenery through the camera of a smartphone. AR glasses are devices that, when worn by the user, display the work procedure in their field of view and can be operated hands-free.The displayed work procedures are presented visually and clearly so that users can easily understand them. The alert unit provides error prevention and mitigation alerts based on the work procedures displayed by the display unit. Error prevention and mitigation alerts are based on, for example, the type of error, the content of the alert, and the timing of its provision, but are not limited to these examples. For example, the alert unit provides an alert if the user makes a mistake in the work procedure. The alert unit can also provide specific correction methods when an error occurs. Furthermore, the alert unit can provide advance warnings to prevent errors. For example, the alert unit notifies the user of an error using audio or visual alerts. The content of error prevention and mitigation alerts is presented specifically and clearly so that users can quickly correct errors. As a result, the AI ​​agent system according to this embodiment can support users in starting and accurately performing tasks in a short amount of time.

[0030] The reception desk receives user input. User input includes, but is not limited to, voice input, touch input, and keyboard input. For example, the reception desk can receive user instructions using voice input. The reception desk can also receive user instructions using touch input. The reception desk can also receive user instructions using keyboard input. For example, the reception desk can analyze the user's voice input using speech recognition technology and receive instructions. Touch input is a method in which the user inputs instructions by operating a touch panel, allowing for intuitive operation. Keyboard input is a method in which the user inputs instructions using a keyboard, allowing for accurate input. The reception desk can also use a combination of these input methods to improve user convenience. For example, by using voice input and touch input together, the user can give instructions by voice while performing detailed operations on the touch panel. Furthermore, the reception desk can record the user's input history and have a function to predict the next input based on past input content. This allows the user to input instructions more quickly and efficiently. Speech recognition technology analyzes the user's voice using natural language processing and accurately understands their intent. For example, if a user says, "Tell me the next step," speech recognition technology analyzes this instruction and instructs the system to display the next step in the process. Touch input allows users to operate by tapping icons or buttons on the screen. For example, if a user taps the "Start" button on the screen, the system will begin guiding users through the process. Keyboard input is a method where users enter text using a keyboard, which is particularly effective for entering detailed instructions or comments. For example, if a user types "Tell me more about this step" on the keyboard, the system will display a detailed explanation of the relevant step. This allows the reception desk to support diverse input methods and enable flexible operation tailored to user needs.

[0031] The guidance unit provides voice guidance based on information received by the reception unit. Voice guidance may vary depending on factors such as the type of voice, the content of the guidance, and the timing of its delivery. For example, the guidance unit may provide voice instructions for the user on the basic procedures of a task. It can also provide voice instructions for information necessary for the user to begin a task. Furthermore, if the user makes a mistake in the procedure, the guidance unit can guide them through the correct procedure via voice. For example, the guidance unit may use speech synthesis technology to provide guidance in a natural-sounding voice. The voice guidance is explained in clear and easy-to-understand language to help the user understand the task. The content of the guidance is appropriately adjusted according to the type and situation of the task. The guidance unit can adjust the content and pace of the guidance according to the user's level of understanding and progress. For example, it may provide detailed instructions for a first-time user and concise guidance for experienced users. The guidance unit can also improve the guidance content based on user feedback. For example, if a user provides feedback that "this explanation is difficult to understand," the guidance unit will review the explanation and change it to a clearer expression. Furthermore, the guidance unit can support multiple languages, providing appropriate guidance to users who speak different languages. For example, it can improve comprehension by providing guidance in the user's native language, such as English, Spanish, or Chinese. Speech synthesis technology achieves natural pronunciation and intonation, providing voice guidance that is easy for users to understand. As a result, the guidance unit can support users in accurately understanding and efficiently performing their tasks.

[0032] The display unit displays work procedures in AR based on audio guidance provided by the guidance unit. AR display is performed using, for example, a smartphone or AR glasses, but is not limited to these examples. For instance, the display unit displays work procedures on a smartphone screen. The display unit can also visually demonstrate work procedures using AR glasses. Furthermore, the display unit can provide detailed explanations of each step of the work procedure. For example, the display unit overlays work procedures onto the real-world scenery via a smartphone camera. AR glasses are devices worn by the user that display work procedures in their field of view and allow for hands-free operation. The displayed work procedures are presented visually in an easy-to-understand manner for the user. The display unit can track the user's gaze and movements to display work procedures at the appropriate time. For example, when the user looks at a specific work area, it displays the procedure related to that area. The display unit can also update its display content according to the user's progress, indicating the next step. For example, it can automatically display the next step when the user completes a certain step. Additionally, the display unit can provide detailed explanations and diagrams of the work procedures. For example, when demonstrating how to install a specific part, the system displays detailed diagrams of the part and installation instructions. This makes it easier for users to visually understand the procedure, enabling them to perform the work accurately. By utilizing AR technology, the display overlays the work procedures onto the real-world work environment, allowing users to intuitively understand the steps. In this way, the display can support users in efficiently performing their tasks.

[0033] The alert unit provides error prevention and mitigation alerts based on the work procedures displayed by the display unit. Error prevention and mitigation alerts are based on, for example, the type of error, the content of the alert, and the timing of its provision, but are not limited to these examples. For example, the alert unit provides an alert when a user makes a mistake in the work procedure. The alert unit can also provide specific corrective methods when an error occurs. Furthermore, the alert unit can provide advance warnings to prevent errors. For example, the alert unit can notify the user of an error using audio or visual alerts. The content of error prevention and mitigation alerts is presented in a specific and easy-to-understand manner so that the user can quickly correct the error. The alert unit can monitor user operations in real time and provide preventive alerts before errors occur. For example, if a user attempts to perform an incorrect procedure, it can immediately issue an alert and guide the user to the correct procedure. The alert unit can also analyze past error data and provide preventive measures for frequently occurring errors. For example, if errors frequently occur in a particular procedure, it can strengthen the warnings for that procedure. In addition, the alert unit can improve the content of alerts based on user feedback. For example, if a user provides feedback that "this alert is unclear," the alerting system will review the content and change it to a clearer expression. Alerts are provided through a variety of means, such as audio, visual, and vibration, to ensure that users are reliably recognized. This allows the alerting system to support users in quickly correcting errors and performing their tasks accurately.

[0034] The guidance unit can provide basic customer service procedures via voice guidance. For example, the guidance unit can explain basic customer service procedures via voice. For example, the guidance unit can provide voice guidance on basic customer service procedures such as greeting customers, explaining products, and operating the cash register. The guidance unit can also explain basic customer service procedures step by step. For example, the guidance unit can explain the procedure for greeting customers via voice, and then sequentially guide the user through the procedures for explaining products and operating the cash register. The guidance unit can also explain basic customer service procedures repeatedly. For example, the guidance unit can provide voice explanations as many times as necessary until the user understands the procedure. This allows users to quickly understand the work by providing basic customer service procedures via voice guidance. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without AI. For example, the guidance unit can use speech synthesis technology to explain basic customer service procedures via voice.

[0035] The display unit can visually show work procedures using a smartphone or AR glasses. For example, the display unit can display work procedures on a smartphone screen. For example, the display unit can overlay work procedures onto a real-world scene using a smartphone's camera. The display unit can also visually show work procedures using AR glasses. For example, the display unit can display work procedures in the field of view of a user wearing AR glasses. The display unit can also show each step of the work procedure in detail. For example, the display unit can sequentially display each step of the work procedure, guiding the user to easily understand the procedure. This makes it easier for users to understand the work procedure by visually showing it. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input video data acquired by a smartphone's camera into a generating AI and have the generating AI execute the display of the work procedure.

[0036] The alert unit can provide alerts when a user makes a mistake in the work procedure. For example, the alert unit can provide an alert such as, "That procedure is incorrect. The correct procedure is here," when a user performs an incorrect procedure. The alert unit can also provide visual alerts when a user makes a mistake in the work procedure. For example, the alert unit can display a warning message on the screen when a user performs an incorrect procedure. The alert unit can also provide alerts using both audio and visual means when a user makes a mistake in the work procedure. For example, the alert unit can provide an audio warning and simultaneously display a warning message on the screen when a user performs an incorrect procedure. This prevents errors by providing alerts when a user makes a mistake in the work procedure. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input user operation data into a generating AI and have the generating AI detect incorrect procedures and provide alerts.

[0037] The guidance unit can provide users with the information they need to start their work. For example, the guidance unit can provide the information users need to start their work via voice. For instance, the guidance unit can provide voice guidance on the overview of the work, points to note, and explanations of the tools to be used as the information users need to start their work. The guidance unit can also provide the information users need to start their work in stages. For example, the guidance unit can explain the overview of the work, and then sequentially provide explanations of points to note and the tools to be used. The guidance unit can also provide the information users need to start their work repeatedly. For example, the guidance unit can provide voice explanations as many times as necessary until the user understands the information. This ensures that users can start their work smoothly by providing them with the information they need to start their work. Some or all of the above processing in the guidance unit may be performed using AI, for example, or not using AI. For example, the guidance unit can use speech synthesis technology to provide the information users need to start their work via voice.

[0038] The display unit can show each step of the work procedure in detail. For example, the display unit can sequentially display each step of the work procedure, guiding the user to easily understand the procedure. For example, the display unit can visually represent each step of the work procedure, supporting the user in accurately performing the procedure. The display unit can also enable the user to accurately perform the task by showing each step of the work procedure in detail. For example, the display unit can guide the user to avoid making mistakes by showing each step of the work procedure in detail. The display unit can also enable the user to efficiently perform the task by showing each step of the work procedure in detail. For example, the display unit can enable the user to quickly understand the procedure by showing each step of the work procedure in detail. As a result, by showing each step of the work procedure in detail, the user can accurately perform the task. Some or all of the above processing in the display unit may be performed using AI, for example, or not using AI. For example, the display unit can use generative AI to show each step of the work procedure in detail.

[0039] The alert unit can provide specific corrective actions when an error occurs. For example, the alert unit can provide specific corrective actions when an error occurs. For example, when a user performs an incorrect procedure, the alert unit can provide specific corrective actions such as, "That procedure is incorrect. The correct procedure is here." The alert unit can also provide visual corrective actions when an error occurs. For example, when a user performs an incorrect procedure, the alert unit can display the corrective action on the screen. The alert unit can also provide corrective actions both audibly and visually when an error occurs. For example, when a user performs an incorrect procedure, the alert unit can provide verbal guidance on how to correct the error and simultaneously display the corrective action on the screen. This allows users to quickly correct errors by providing specific corrective actions when they occur. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input user operation data into a generating AI and have the generating AI perform error detection and provide corrective actions.

[0040] The reception desk can analyze the user's past input history and select the most suitable reception method. For example, if the user has preferred using voice input in the past, the reception desk will prioritize suggesting voice input. For example, if the user has preferred using voice input in the past, the reception desk will ask, "Would you like to use voice input?" The reception desk can also prioritize suggesting text input if the user has frequently used text input in the past. For example, if the user has frequently used text input in the past, the reception desk will ask, "Would you like to use text input?" The reception desk can also adjust the reception process to match the time period in which the user has previously entered data. For example, if the user has previously entered data in a specific time period, the reception desk will ask, "Would you like to start entering data?" by voice at that time period. In this way, by analyzing the user's past input history, the reception desk can provide the most suitable reception method. Some or all of the above processing in the reception desk may be performed using AI, for example, or not. For example, the reception desk can input the user's past input history data into a generating AI and have the AI ​​select the optimal reception method.

[0041] The reception desk can filter the user's current situation and areas of interest when receiving input. For example, the reception desk can display only relevant tasks based on the user's current situation. For example, the reception desk can announce to the user via voice, "We will now display only relevant tasks." The reception desk can also prioritize the display of relevant tasks based on the user's areas of interest. For example, the reception desk can announce via voice, "We will now prioritize the display of relevant tasks." The reception desk can also filter out unnecessary information based on the user's current situation and areas of interest. For example, the reception desk can announce via voice, "We will now filter out unnecessary information." This allows the reception desk to prioritize the provision of relevant information by filtering based on the user's current situation and areas of interest. Some or all of the above processing in the reception desk may be performed using AI, for example, or without AI. For example, the reception desk can input data on the user's current situation and areas of interest into a generating AI and have the generating AI perform the filtering.

[0042] The reception desk can prioritize receiving highly relevant information by considering the user's geographical location when receiving input. For example, if the user is in a specific region, the reception desk will prioritize receiving tasks related to that region. For example, if the user is in a specific region, the reception desk will announce via voice, "We will prioritize receiving tasks related to that region." The reception desk can also prioritize receiving relevant tasks based on the user's current location if the user is on the move. For example, if the reception desk is on the move, the reception desk will announce via voice, "We will prioritize receiving tasks related to your current location." The reception desk can also prioritize receiving information related to a specific location if the user is in a specific location. For example, if the reception desk is in a specific location, the reception desk will announce via voice, "We will prioritize receiving information related to that location." In this way, by considering the user's geographical location, highly relevant information can be prioritized. Some or all of the above processing in the reception desk may be performed using AI, for example, or without AI. For example, the reception desk can input the user's geographical location data into a generating AI and have the AI ​​select highly relevant information.

[0043] The reception desk can analyze the user's social media activity and receive relevant information when data is received. For example, the reception desk can prioritize tasks related to topics the user has shown interest in on social media. For example, the reception desk can announce via voice, "We will prioritize tasks related to topics the user has shown interest in on social media." The reception desk can also suggest relevant tasks based on the user's social media activity. For example, the reception desk can analyze the user's social media activity and announce via voice, "We will suggest relevant tasks." The reception desk can also analyze the user's social media activity and suggest the most suitable tasks. For example, the reception desk can analyze the user's social media activity and announce via voice, "We will suggest the most suitable tasks." This allows the reception desk to prioritize receiving relevant information by analyzing the user's social media activity. Some or all of the above processing in the reception desk may be performed using AI, for example, or not. For example, the reception desk can input the user's social media activity data into a generating AI and have the generating AI select relevant information.

[0044] The guidance unit can adjust the level of detail of the guidance based on the importance of the task when providing guidance. For example, if the task is important, the guidance unit will provide detailed guidance. For example, if the task is important, the guidance unit will announce in voice, "This task is important, so we will provide detailed guidance," and then provide detailed guidance. The guidance unit can also provide concise guidance for simple tasks. For example, if the task is simple, the guidance unit will announce in voice, "This task is simple, so we will provide concise guidance," and then provide concise guidance. The guidance unit can also adjust the level of detail of the guidance according to the importance of the task. For example, depending on the importance of the task, the guidance unit will announce in voice, "We will adjust the level of detail of the guidance according to the importance of the task," and then provide guidance with an appropriate level of detail. In this way, appropriate guidance can be provided by adjusting the level of detail of the guidance based on the importance of the task. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without using AI. For example, the guidance unit can input task importance data into a generating AI and have the generating AI perform the adjustment of the level of detail of the guidance.

[0045] The guidance unit can apply different guidance algorithms depending on the category of work when providing guidance. For example, in the case of customer service, the guidance unit provides customer service guidance. For example, in the case of customer service, the guidance unit will announce "We will provide customer service guidance" and then provide customer service guidance. The guidance unit can also provide data entry guidance in the case of clerical work. For example, in the case of clerical work, the guidance unit will announce "We will provide data entry guidance" and then provide data entry guidance. Furthermore, the guidance unit can apply the most suitable guidance algorithm depending on the category of work. For example, depending on the category of work, the guidance unit will announce "We will apply the most suitable guidance algorithm according to the category of work" and then provide appropriate guidance. This allows users to accurately perform their tasks by applying the most suitable guidance algorithm according to the category of work. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without using AI. For example, the guidance unit can input the category of work data into a generating AI and have the generating AI execute the application of the guidance algorithm.

[0046] The guidance unit can determine the priority of guidance based on the start date of the work when providing guidance. For example, if the start date of the work is approaching, the guidance unit will provide guidance preferentially. For example, if the start date of the work is approaching, the guidance unit will announce via voice, "Since the start date of the work is approaching, we will provide guidance preferentially," and then provide the guidance preferentially. The guidance unit can also postpone providing guidance if the start date of the work is far away. For example, if the start date of the work is far away, the guidance unit will announce via voice, "Since the start date of the work is far away, we will provide guidance later," and then provide the guidance later. The guidance unit can also determine the priority of guidance based on the start date of the work. For example, based on the start date of the work, the guidance unit will announce via voice, "We will determine the priority of guidance based on the start date of the work," and then provide the guidance at the appropriate time. In this way, by determining the priority of guidance based on the start date of the work, guidance can be provided at the appropriate time. Some or all of the above processing in the guidance unit may be performed using AI, for example, or not using AI. For example, the guidance unit can input data on the start date of operations into the generating AI, and have the generating AI determine the priority of the guidance.

[0047] The guidance unit can adjust the order of guidance based on the relevance of the tasks when providing guidance. For example, the guidance unit can provide guidance in order of relevance, starting with the most relevant tasks. For example, the guidance unit can provide guidance by announcing, "We will provide guidance in order of relevance," starting with the most relevant tasks. The guidance unit can also postpone providing guidance for less relevant tasks. For example, the guidance unit can postpone less relevant tasks by announcing, "We will postpone providing guidance for less relevant tasks," and then provide guidance. The guidance unit can also adjust the order of guidance based on the relevance of the tasks. For example, the guidance unit can provide guidance in an appropriate order by announcing, "We will adjust the order of guidance based on the relevance of the tasks," based on the relevance of the tasks. This allows for efficient guidance provision by adjusting the order of guidance based on the relevance of the tasks. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without AI. For example, the guidance unit can input task relevance data into a generating AI and have the generating AI perform the adjustment of the guidance order.

[0048] The display unit can adjust the level of detail displayed based on the importance of the work procedure. For example, if the work procedure is important, the display unit will provide a detailed display. For example, if the work procedure is important, the display unit will announce in voice, "This is an important work procedure, so we will provide a detailed display," and then provide a detailed display. The display unit can also provide a concise display for simple work procedures. For example, if the work procedure is simple, the display unit will announce in voice, "This is a simple work procedure, so we will provide a concise display," and then provide a concise display. The display unit can also adjust the level of detail displayed according to the importance of the work procedure. For example, depending on the importance of the work procedure, the display unit will announce in voice, "The level of detail displayed will be adjusted according to the importance of the work procedure," and then provide a display with an appropriate level of detail. In this way, by adjusting the level of detail displayed based on the importance of the work procedure, an appropriate display can be provided. Some or all of the above processing in the display unit may be performed using AI, for example, or without using AI. For example, the display unit can input importance data for business procedures into a generating AI, and have the generating AI adjust the level of detail displayed.

[0049] The display unit can apply different display algorithms depending on the category of the work procedure when displaying information. For example, in the case of customer service, the display unit can provide a display of customer service. For example, in the case of customer service, the display unit can announce "We will now provide a display of customer service" and then provide a display of customer service. The display unit can also provide a display of data entry in the case of office work. For example, in the case of office work, the display unit can announce "We will now provide a display of data entry" and then provide a display of data entry. Furthermore, the display unit can apply the most suitable display algorithm depending on the category of the work procedure. For example, depending on the category of the work procedure, the display unit can announce "We will now apply the most suitable display algorithm according to the category of the work procedure" and then provide an appropriate display. This allows the user to accurately perform their tasks by applying the most suitable display algorithm according to the category of the work procedure. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the category data of the work procedure into a generating AI and have the generating AI execute the application of the display algorithm.

[0050] The display unit can determine the display priority based on the start time of the business procedure. For example, if the start time of the business procedure is approaching, the display unit will prioritize displaying it. For example, if the start time of the business procedure is approaching, the display unit will announce by voice, "The start time of the business procedure is approaching, so we will display it first," and then display it first. The display unit can also postpone displaying the business procedure if the start time is far off. For example, if the start time of the business procedure is far off, the display unit will announce by voice, "The start time of the business procedure is far off, so we will display it first," and then display it first. The display unit can also determine the display priority based on the start time of the business procedure. For example, based on the start time of the business procedure, the display unit will announce by voice, "We will determine the display priority based on the start time of the business procedure," and then provide the display at the appropriate time. This allows the display to be provided at the appropriate time by determining the display priority based on the start time of the business procedure. Some or all of the above processing in the display unit may be performed using AI, for example, or without using AI. For example, the display unit can input data on the start time of business procedures into the generating AI, and have the generating AI determine the priority of the display.

[0051] The display unit can adjust the display order based on the relevance of the work procedures during display. For example, the display unit can display work procedures in order of relevance. For example, the display unit can provide a voice announcement saying, "We will now display the work procedures in order of relevance," and then display the work procedures in order of relevance. The display unit can also postpone the display of less relevant work procedures. For example, the display unit can postpone the display of less relevant work procedures and provide a voice announcement saying, "We will now postpone the display of less relevant work procedures." The display unit can also adjust the display order based on the relevance of the work procedures. For example, the display unit can provide a voice announcement saying, "We will adjust the display order based on the relevance of the work procedures," and then display the work procedures in the appropriate order. This allows for efficient display by adjusting the display order based on the relevance of the work procedures. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the relevance data of the work procedures into a generating AI and have the generating AI perform the adjustment of the display order.

[0052] The alert unit can adjust the level of detail of an alert based on the severity of the error when providing an alert. For example, in the case of a critical error, the alert unit provides a detailed alert. For example, in the case of a critical error, the alert unit will announce in voice, "Due to a critical error, a detailed alert will be provided," and then provide a detailed alert. The alert unit can also provide a concise alert in the case of a minor error. For example, in the case of a minor error, the alert unit will announce in voice, "Due to a minor error, a concise alert will be provided," and then provide a concise alert. The alert unit can also adjust the level of detail of an alert according to the severity of the error. For example, depending on the severity of the error, the alert unit will announce in voice, "The level of detail of the alert will be adjusted according to the severity of the error," and then provide an alert with the appropriate level of detail. In this way, by adjusting the level of detail of the alert based on the severity of the error, an appropriate alert can be provided. Some or all of the above processing in the alert unit may be performed using AI, for example, or without using AI. For example, the alert unit can input error severity data into a generating AI and have the generating AI perform the adjustment of the level of detail of the alert.

[0053] The alert unit can apply different alert algorithms depending on the error category when providing an alert. For example, in the case of a system error, the alert unit provides a technical alert. For example, in the case of a system error, the alert unit will announce, "Due to a system error, we will provide a technical alert," and then provide the technical alert. The alert unit can also provide an alert indicating how to perform the operation in the case of a user operation error. For example, in the case of a user operation error, the alert unit will announce, "Due to a user operation error, we will provide an alert indicating how to perform the operation," and then provide the alert indicating how to perform the operation. The alert unit can also apply the most suitable alert algorithm depending on the error category. For example, depending on the error category, the alert unit will announce, "We will apply the most suitable alert algorithm according to the error category," and then provide the appropriate alert. This allows users to quickly correct errors by applying the most suitable alert algorithm according to the error category. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input error category data into a generating AI and have the generating AI execute the application of the alert algorithm.

[0054] The alert unit can determine the priority of alerts based on when an error occurs. For example, if the timing of an error is imminent, the alert unit will prioritize providing the alert. For example, if the timing of an error is imminent, the alert unit will announce, "Since the timing of an error is imminent, we will prioritize providing the alert," and then provide the alert. The alert unit can also postpone providing the alert if the timing of an error is far off. For example, if the timing of an error is far off, the alert unit will announce, "Since the timing of an error is far off, we will postpone providing the alert," and then provide the alert. The alert unit can also determine the priority of alerts based on when the error occurs. For example, based on when the error occurs, the alert unit will announce, "We will determine the priority of alerts based on when the error occurred," and then provide the alert at the appropriate time. This allows for timely alert delivery by prioritizing alerts based on when the error occurs. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input error occurrence data into the generating AI and have the generating AI determine the priority of alerts.

[0055] The alert unit can adjust the order of alerts based on the relevance of the errors when providing alerts. For example, the alert unit can provide alerts in order of relevance, starting with the most relevant errors. For example, the alert unit can provide alerts in order of relevance, announcing, "We will provide alerts in order of relevance," and then provide the alerts. The alert unit can also postpone providing alerts for less relevant errors. For example, the alert unit can postpone providing alerts for less relevant errors, announcing, "We will postpone providing alerts for less relevant errors," and then provide the alerts. The alert unit can also adjust the order of alerts based on the relevance of the errors. For example, the alert unit can provide alerts in the appropriate order, announcing, "We will adjust the order of alerts based on the relevance of the errors," and then providing the alerts in the appropriate order. This allows for efficient alert delivery by adjusting the order of alerts based on the relevance of the errors. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input error relevance data into a generating AI and have the generating AI perform the adjustment of the alert order.

[0056] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0057] The reception desk can analyze a user's past input history and select the most suitable reception method. For example, if a user has preferred using voice input in the past, it will prioritize suggesting voice input. The reception desk will then verbally ask, "Would you like to use voice input?" if the user has preferred using voice input in the past. It can also prioritize suggesting text input if the user has frequently used text input in the past. The reception desk will then verbally ask, "Would you like to use text input?" if the user has frequently used text input in the past. Furthermore, if a user has previously entered data during a specific time period, the reception desk can adjust the reception process to match that time period. The reception desk will then verbally ask, "Would you like to start entering data?" if the user has previously entered data during a specific time period. In this way, by analyzing a user's past input history, the system can provide the most suitable reception method.

[0058] The guidance unit can adjust the level of detail in the guidance provided based on the importance of the task. For example, for important tasks, it provides detailed guidance. For important tasks, the guidance unit will announce, "This task is important, so we will provide detailed guidance," and then provide detailed guidance. For simple tasks, it can also provide concise guidance. For simple tasks, the guidance unit will announce, "This task is simple, so we will provide concise guidance," and then provide concise guidance. Furthermore, it can adjust the level of detail in the guidance according to the importance of the task. Depending on the importance of the task, the guidance unit will announce, "We will adjust the level of detail in the guidance according to the importance of the task," and then provide guidance with the appropriate level of detail. This allows for the provision of appropriate guidance by adjusting the level of detail based on the importance of the task.

[0059] The display unit can adjust the level of detail displayed based on the importance of the work procedure. For example, for important work procedures, it provides a detailed display. For important work procedures, the display unit will announce, "This is an important work procedure, so we will provide a detailed display," and then provide the detailed display. For simple work procedures, it can also provide a concise display. For simple work procedures, the display unit will announce, "This is a simple work procedure, so we will provide a concise display," and then provide the concise display. Furthermore, it can adjust the level of detail displayed according to the importance of the work procedure. Depending on the importance of the work procedure, the display unit will announce, "The level of detail displayed will be adjusted according to the importance of the work procedure," and then provide a display with an appropriate level of detail. In this way, by adjusting the level of detail displayed based on the importance of the work procedure, an appropriate display can be provided.

[0060] The alerting unit can adjust the level of detail of an alert based on the severity of the error when providing an alert. For example, in the case of a critical error, it provides a detailed alert. In the case of a critical error, the alerting unit will announce, "Due to a critical error, a detailed alert will be provided," and then provide a detailed alert. It can also provide a concise alert in the case of a minor error. In the case of a minor error, the alerting unit will announce, "Due to a minor error, a concise alert will be provided," and then provide a concise alert. Furthermore, it can adjust the level of detail of an alert according to the severity of the error. Depending on the severity of the error, the alerting unit will announce, "The level of detail of the alert will be adjusted according to the severity of the error," and then provide an alert with the appropriate level of detail. In this way, by adjusting the level of detail of the alert based on the severity of the error, appropriate alerts can be provided.

[0061] The reception desk can prioritize receiving highly relevant information by considering the user's geographical location during the input process. For example, if a user is in a specific region, it will prioritize receiving tasks related to that region. The reception desk will announce to the user via voice, "We will prioritize receiving tasks related to that region." Furthermore, if a user is on the move, it can prioritize receiving tasks related to their current location. The reception desk will announce to the user via voice, "We will prioritize receiving tasks related to your current location." Additionally, if a user is in a specific location, it can prioritize receiving information related to that location. The reception desk will announce to the user via voice, "We will prioritize receiving information related to that location." This allows the reception desk to prioritize receiving highly relevant information by considering the user's geographical location.

[0062] The guidance unit can prioritize guidance based on the start date of the work when providing guidance. For example, if the start date of the work is approaching, guidance will be provided preferentially. If the start date of the work is approaching, the guidance unit will announce via voice, "Since the start date of the work is approaching, we will provide guidance preferentially," and then provide the guidance preferentially. Conversely, if the start date of the work is far off, guidance can be postponed. If the start date of the work is far off, the guidance unit will announce via voice, "Since the start date of the work is far off, we will provide guidance later," and then provide the guidance later. The guidance unit can also determine the priority of guidance based on the start date of the work. Based on the start date of the work, the guidance unit will announce via voice, "We will determine the priority of guidance based on the start date of the work," and provide guidance at the appropriate time. This ensures that guidance is provided at the appropriate time by determining the priority of guidance based on the start date of the work.

[0063] The following briefly describes the processing flow for example form 1.

[0064] Step 1: The reception desk receives user input. User input includes voice input, touch input, and keyboard input. For example, the reception desk uses voice recognition technology to analyze the user's voice input and receive instructions. Touch input is a method in which the user enters instructions by operating a touch panel, allowing for intuitive operation. Keyboard input is a method in which the user enters instructions using a keyboard, allowing for accurate input. Step 2: The guidance unit provides voice guidance based on the information received by the reception unit. The voice guidance is provided based on the type of voice, the content of the guidance, and the timing of its provision. For example, the guidance unit explains the basic procedures of the work to the user via voice. It also provides information necessary for the user to start the work via voice, and if the user makes a mistake in the work procedure, it guides them to the correct procedure via voice. Step 3: The display unit displays the work procedure in AR based on the audio guidance provided by the guidance unit. The AR display is performed using a smartphone or AR glasses. For example, the display unit can display the work procedure on a smartphone screen, and the work procedure can be visually shown using AR glasses. Each step of the work procedure is shown in detail and displayed in a visually easy-to-understand manner for the user. Step 4: The alert unit provides error prevention and mitigation alerts based on the work procedures displayed by the display unit. Error prevention and mitigation alerts are provided based on the type of error, the content of the alert, and the timing of its provision. For example, the alert unit provides an alert if the user makes a mistake in the work procedure, and provides specific corrective methods when an error occurs. It can also provide advance warnings to prevent errors.

[0065] (Example of form 2) The AI ​​agent system according to an embodiment of the present invention is a system for starting work quickly in today's world where the demand for part-time jobs and side jobs is increasing. This system explains the basic procedures of a task to the user through voice guidance, visually displays the procedures using AR display, and provides error prevention and prevention alerts. For example, when a user starts work, the AI ​​agent system explains the basic procedures of the task through voice guidance. For example, in the case of customer service, the AI ​​agent system provides voice guidance such as, "I'll teach you how to serve customers in this section. Our best-selling products are over here. If you have any questions, feel free to ask anytime." Next, the AI ​​agent system visually displays the procedures using AR display. For example, the user can check the procedures using a smartphone or AR glasses. This makes it easier for the user to visually understand the procedures. Furthermore, the AI ​​agent system provides error prevention and prevention alerts to support the user in accurately performing the task. For example, if the user makes a mistake in the procedure, the AI ​​agent system provides an alert such as, "That procedure is incorrect. The correct procedure is here." This mechanism allows work to be started quickly, and enables efficient work even when it is difficult to secure training time or when it takes time to understand the work manual. Potential target users include company employees, students, housewives / househusbands, and freelancers seeking side jobs. This would allow the AI ​​agent system to support users in starting and accurately completing tasks in a short amount of time.

[0066] The AI ​​agent system according to this embodiment comprises a reception unit, a guidance unit, a display unit, and an alert unit. The reception unit receives user input. User input includes, but is not limited to, voice input, touch input, and keyboard input. The reception unit receives user instructions using, for example, voice input. The reception unit can also receive user instructions using touch input. The reception unit can also receive user instructions using keyboard input. For example, the reception unit analyzes the user's voice input using speech recognition technology and receives instructions. Touch input is a method in which the user inputs instructions by operating a touch panel, allowing for intuitive operation. Keyboard input is a method in which the user inputs instructions using a keyboard, allowing for accurate input. The guidance unit provides voice guidance based on the information received by the reception unit. Voice guidance is provided based on, for example, the type of voice, the content of the guidance, the timing of provision, etc., but is not limited to these examples. For example, the guidance unit explains the basic procedures of the work to the user by voice. The guidance unit can also provide information necessary for the user to start the work by voice. Furthermore, the guidance unit can provide voice guidance to correct the procedure if the user makes a mistake in the work procedure. For example, the guidance unit can provide guidance in a natural voice using speech synthesis technology. The voice guidance is explained in easy-to-understand language so that the user can easily understand the work. The content of the guidance is appropriately adjusted according to the type and situation of the work. The display unit displays the work procedure in AR based on the voice guidance provided by the guidance unit. The AR display is performed using, for example, a smartphone or AR glasses, but is not limited to these examples. For example, the display unit displays the work procedure on the screen of a smartphone. The display unit can also visually show the work procedure using AR glasses. The display unit can also show each step of the work procedure in detail. For example, the display unit can overlay the work procedure onto the real-world scenery through the camera of a smartphone. AR glasses are devices that, when worn by the user, display the work procedure in their field of view and can be operated hands-free.The displayed work procedures are presented visually and clearly so that users can easily understand them. The alert unit provides error prevention and mitigation alerts based on the work procedures displayed by the display unit. Error prevention and mitigation alerts are based on, for example, the type of error, the content of the alert, and the timing of its provision, but are not limited to these examples. For example, the alert unit provides an alert if the user makes a mistake in the work procedure. The alert unit can also provide specific correction methods when an error occurs. Furthermore, the alert unit can provide advance warnings to prevent errors. For example, the alert unit notifies the user of an error using audio or visual alerts. The content of error prevention and mitigation alerts is presented specifically and clearly so that users can quickly correct errors. As a result, the AI ​​agent system according to this embodiment can support users in starting and accurately performing tasks in a short amount of time.

[0067] The reception desk receives user input. User input includes, but is not limited to, voice input, touch input, and keyboard input. For example, the reception desk can receive user instructions using voice input. The reception desk can also receive user instructions using touch input. The reception desk can also receive user instructions using keyboard input. For example, the reception desk can analyze the user's voice input using speech recognition technology and receive instructions. Touch input is a method in which the user inputs instructions by operating a touch panel, allowing for intuitive operation. Keyboard input is a method in which the user inputs instructions using a keyboard, allowing for accurate input. The reception desk can also use a combination of these input methods to improve user convenience. For example, by using voice input and touch input together, the user can give instructions by voice while performing detailed operations on the touch panel. Furthermore, the reception desk can record the user's input history and have a function to predict the next input based on past input content. This allows the user to input instructions more quickly and efficiently. Speech recognition technology analyzes the user's voice using natural language processing and accurately understands their intent. For example, if a user says, "Tell me the next step," speech recognition technology analyzes this instruction and instructs the system to display the next step in the process. Touch input allows users to operate by tapping icons or buttons on the screen. For example, if a user taps the "Start" button on the screen, the system will begin guiding users through the process. Keyboard input is a method where users enter text using a keyboard, which is particularly effective for entering detailed instructions or comments. For example, if a user types "Tell me more about this step" on the keyboard, the system will display a detailed explanation of the relevant step. This allows the reception desk to support diverse input methods and enable flexible operation tailored to user needs.

[0068] The guidance unit provides voice guidance based on information received by the reception unit. Voice guidance may vary depending on factors such as the type of voice, the content of the guidance, and the timing of its delivery. For example, the guidance unit may provide voice instructions for the user on the basic procedures of a task. It can also provide voice instructions for information necessary for the user to begin a task. Furthermore, if the user makes a mistake in the procedure, the guidance unit can guide them through the correct procedure via voice. For example, the guidance unit may use speech synthesis technology to provide guidance in a natural-sounding voice. The voice guidance is explained in clear and easy-to-understand language to help the user understand the task. The content of the guidance is appropriately adjusted according to the type and situation of the task. The guidance unit can adjust the content and pace of the guidance according to the user's level of understanding and progress. For example, it may provide detailed instructions for a first-time user and concise guidance for experienced users. The guidance unit can also improve the guidance content based on user feedback. For example, if a user provides feedback that "this explanation is difficult to understand," the guidance unit will review the explanation and change it to a clearer expression. Furthermore, the guidance unit can support multiple languages, providing appropriate guidance to users who speak different languages. For example, it can improve comprehension by providing guidance in the user's native language, such as English, Spanish, or Chinese. Speech synthesis technology achieves natural pronunciation and intonation, providing voice guidance that is easy for users to understand. As a result, the guidance unit can support users in accurately understanding and efficiently performing their tasks.

[0069] The display unit displays work procedures in AR based on audio guidance provided by the guidance unit. AR display is performed using, for example, a smartphone or AR glasses, but is not limited to these examples. For instance, the display unit displays work procedures on a smartphone screen. The display unit can also visually demonstrate work procedures using AR glasses. Furthermore, the display unit can provide detailed explanations of each step of the work procedure. For example, the display unit overlays work procedures onto the real-world scenery via a smartphone camera. AR glasses are devices worn by the user that display work procedures in their field of view and allow for hands-free operation. The displayed work procedures are presented visually in an easy-to-understand manner for the user. The display unit can track the user's gaze and movements to display work procedures at the appropriate time. For example, when the user looks at a specific work area, it displays the procedure related to that area. The display unit can also update its display content according to the user's progress, indicating the next step. For example, it can automatically display the next step when the user completes a certain step. Additionally, the display unit can provide detailed explanations and diagrams of the work procedures. For example, when demonstrating how to install a specific part, the system displays detailed diagrams of the part and installation instructions. This makes it easier for users to visually understand the procedure, enabling them to perform the work accurately. By utilizing AR technology, the display overlays the work procedures onto the real-world work environment, allowing users to intuitively understand the steps. In this way, the display can support users in efficiently performing their tasks.

[0070] The alert unit provides error prevention and mitigation alerts based on the work procedures displayed by the display unit. Error prevention and mitigation alerts are based on, for example, the type of error, the content of the alert, and the timing of its provision, but are not limited to these examples. For example, the alert unit provides an alert when a user makes a mistake in the work procedure. The alert unit can also provide specific corrective methods when an error occurs. Furthermore, the alert unit can provide advance warnings to prevent errors. For example, the alert unit can notify the user of an error using audio or visual alerts. The content of error prevention and mitigation alerts is presented in a specific and easy-to-understand manner so that the user can quickly correct the error. The alert unit can monitor user operations in real time and provide preventive alerts before errors occur. For example, if a user attempts to perform an incorrect procedure, it can immediately issue an alert and guide the user to the correct procedure. The alert unit can also analyze past error data and provide preventive measures for frequently occurring errors. For example, if errors frequently occur in a particular procedure, it can strengthen the warnings for that procedure. In addition, the alert unit can improve the content of alerts based on user feedback. For example, if a user provides feedback that "this alert is unclear," the alerting system will review the content and change it to a clearer expression. Alerts are provided through a variety of means, such as audio, visual, and vibration, to ensure that users are reliably recognized. This allows the alerting system to support users in quickly correcting errors and performing their tasks accurately.

[0071] The guidance unit can provide basic customer service procedures via voice guidance. For example, the guidance unit can explain basic customer service procedures via voice. For example, the guidance unit can provide voice guidance on basic customer service procedures such as greeting customers, explaining products, and operating the cash register. The guidance unit can also explain basic customer service procedures step by step. For example, the guidance unit can explain the procedure for greeting customers via voice, and then sequentially guide the user through the procedures for explaining products and operating the cash register. The guidance unit can also explain basic customer service procedures repeatedly. For example, the guidance unit can provide voice explanations as many times as necessary until the user understands the procedure. This allows users to quickly understand the work by providing basic customer service procedures via voice guidance. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without AI. For example, the guidance unit can use speech synthesis technology to explain basic customer service procedures via voice.

[0072] The display unit can visually show work procedures using a smartphone or AR glasses. For example, the display unit can display work procedures on a smartphone screen. For example, the display unit can overlay work procedures onto a real-world scene using a smartphone's camera. The display unit can also visually show work procedures using AR glasses. For example, the display unit can display work procedures in the field of view of a user wearing AR glasses. The display unit can also show each step of the work procedure in detail. For example, the display unit can sequentially display each step of the work procedure, guiding the user to easily understand the procedure. This makes it easier for users to understand the work procedure by visually showing it. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input video data acquired by a smartphone's camera into a generating AI and have the generating AI execute the display of the work procedure.

[0073] The alert unit can provide alerts when a user makes a mistake in the work procedure. For example, the alert unit can provide an alert such as, "That procedure is incorrect. The correct procedure is here," when a user performs an incorrect procedure. The alert unit can also provide visual alerts when a user makes a mistake in the work procedure. For example, the alert unit can display a warning message on the screen when a user performs an incorrect procedure. The alert unit can also provide alerts using both audio and visual means when a user makes a mistake in the work procedure. For example, the alert unit can provide an audio warning and simultaneously display a warning message on the screen when a user performs an incorrect procedure. This prevents errors by providing alerts when a user makes a mistake in the work procedure. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input user operation data into a generating AI and have the generating AI detect incorrect procedures and provide alerts.

[0074] The guidance unit can provide users with the information they need to start their work. For example, the guidance unit can provide the information users need to start their work via voice. For instance, the guidance unit can provide voice guidance on the overview of the work, points to note, and explanations of the tools to be used as the information users need to start their work. The guidance unit can also provide the information users need to start their work in stages. For example, the guidance unit can explain the overview of the work, and then sequentially provide explanations of points to note and the tools to be used. The guidance unit can also provide the information users need to start their work repeatedly. For example, the guidance unit can provide voice explanations as many times as necessary until the user understands the information. This ensures that users can start their work smoothly by providing them with the information they need to start their work. Some or all of the above processing in the guidance unit may be performed using AI, for example, or not using AI. For example, the guidance unit can use speech synthesis technology to provide the information users need to start their work via voice.

[0075] The display unit can show each step of the work procedure in detail. For example, the display unit can sequentially display each step of the work procedure, guiding the user to easily understand the procedure. For example, the display unit can visually represent each step of the work procedure, supporting the user in accurately performing the procedure. The display unit can also enable the user to accurately perform the task by showing each step of the work procedure in detail. For example, the display unit can guide the user to avoid making mistakes by showing each step of the work procedure in detail. The display unit can also enable the user to efficiently perform the task by showing each step of the work procedure in detail. For example, the display unit can enable the user to quickly understand the procedure by showing each step of the work procedure in detail. As a result, by showing each step of the work procedure in detail, the user can accurately perform the task. Some or all of the above processing in the display unit may be performed using AI, for example, or not using AI. For example, the display unit can use generative AI to show each step of the work procedure in detail.

[0076] The alert unit can provide specific corrective actions when an error occurs. For example, the alert unit can provide specific corrective actions when an error occurs. For example, when a user performs an incorrect procedure, the alert unit can provide specific corrective actions such as, "That procedure is incorrect. The correct procedure is here." The alert unit can also provide visual corrective actions when an error occurs. For example, when a user performs an incorrect procedure, the alert unit can display the corrective action on the screen. The alert unit can also provide corrective actions both audibly and visually when an error occurs. For example, when a user performs an incorrect procedure, the alert unit can provide verbal guidance on how to correct the error and simultaneously display the corrective action on the screen. This allows users to quickly correct errors by providing specific corrective actions when they occur. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input user operation data into a generating AI and have the generating AI perform error detection and provide corrective actions.

[0077] The reception desk can estimate the user's emotions and adjust the timing of input acceptance based on the estimated emotions. For example, if the user is nervous, the reception desk can prompt them to take some time to relax before prompting them to enter data. For example, if the user is nervous, the reception desk can provide voice guidance saying, "Please relax a little before entering data." The reception desk can also immediately begin accepting data if the user is in a hurry to prompt them to enter data quickly. For example, if the user is in a hurry, the reception desk can provide voice guidance saying, "Please start entering data immediately." The reception desk can also suggest a break if the user is tired and prompt them to enter data after the break. For example, if the user is tired, the reception desk can provide voice guidance saying, "Please take a short break before entering data." By adjusting the timing of input acceptance according to the user's emotions, the system allows users to input data comfortably. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the reception area may be performed using AI, for example, or without AI. For example, the reception area can input user emotion data into a generating AI and have the generating AI perform emotion estimation and adjust the timing of input reception.

[0078] The reception desk can analyze the user's past input history and select the most suitable reception method. For example, if the user has preferred using voice input in the past, the reception desk will prioritize suggesting voice input. For example, if the user has preferred using voice input in the past, the reception desk will ask, "Would you like to use voice input?" The reception desk can also prioritize suggesting text input if the user has frequently used text input in the past. For example, if the user has frequently used text input in the past, the reception desk will ask, "Would you like to use text input?" The reception desk can also adjust the reception process to match the time period in which the user has previously entered data. For example, if the user has previously entered data in a specific time period, the reception desk will ask, "Would you like to start entering data?" by voice at that time period. In this way, by analyzing the user's past input history, the reception desk can provide the most suitable reception method. Some or all of the above processing in the reception desk may be performed using AI, for example, or not. For example, the reception desk can input the user's past input history data into a generating AI and have the AI ​​select the optimal reception method.

[0079] The reception desk can filter the user's current situation and areas of interest when receiving input. For example, the reception desk can display only relevant tasks based on the user's current situation. For example, the reception desk can announce to the user via voice, "We will now display only relevant tasks." The reception desk can also prioritize the display of relevant tasks based on the user's areas of interest. For example, the reception desk can announce via voice, "We will now prioritize the display of relevant tasks." The reception desk can also filter out unnecessary information based on the user's current situation and areas of interest. For example, the reception desk can announce via voice, "We will now filter out unnecessary information." This allows the reception desk to prioritize the provision of relevant information by filtering based on the user's current situation and areas of interest. Some or all of the above processing in the reception desk may be performed using AI, for example, or without AI. For example, the reception desk can input data on the user's current situation and areas of interest into a generating AI and have the generating AI perform the filtering.

[0080] The reception desk can estimate the user's emotions and determine the priority of information to receive based on the estimated emotions. For example, if the user is stressed, the reception desk will prioritize receiving important information. For example, if the user is stressed, the reception desk will announce via voice, "We will prioritize receiving important information." The reception desk can also prioritize receiving detailed information if the user is relaxed. For example, if the user is relaxed, the reception desk will announce via voice, "We will prioritize receiving detailed information." The reception desk can also prioritize receiving information that needs to be processed quickly if the user is in a hurry. For example, if the user is in a hurry, the reception desk will announce via voice, "We will prioritize receiving information that needs to be processed quickly." In this way, by determining the priority of information to receive according to the user's emotions, important information can be prioritized. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the reception area may be performed using AI, for example, or without AI. For example, the reception area can input user emotion data into a generating AI and have the generating AI perform emotion estimation and determine the priority of information.

[0081] The reception desk can prioritize receiving highly relevant information by considering the user's geographical location when receiving input. For example, if the user is in a specific region, the reception desk will prioritize receiving tasks related to that region. For example, if the user is in a specific region, the reception desk will announce via voice, "We will prioritize receiving tasks related to that region." The reception desk can also prioritize receiving relevant tasks based on the user's current location if the user is on the move. For example, if the reception desk is on the move, the reception desk will announce via voice, "We will prioritize receiving tasks related to your current location." The reception desk can also prioritize receiving information related to a specific location if the user is in a specific location. For example, if the reception desk is in a specific location, the reception desk will announce via voice, "We will prioritize receiving information related to that location." In this way, by considering the user's geographical location, highly relevant information can be prioritized. Some or all of the above processing in the reception desk may be performed using AI, for example, or without AI. For example, the reception desk can input the user's geographical location data into a generating AI and have the AI ​​select highly relevant information.

[0082] The reception desk can analyze the user's social media activity and receive relevant information when data is received. For example, the reception desk can prioritize tasks related to topics the user has shown interest in on social media. For example, the reception desk can announce via voice, "We will prioritize tasks related to topics the user has shown interest in on social media." The reception desk can also suggest relevant tasks based on the user's social media activity. For example, the reception desk can analyze the user's social media activity and announce via voice, "We will suggest relevant tasks." The reception desk can also analyze the user's social media activity and suggest the most suitable tasks. For example, the reception desk can analyze the user's social media activity and announce via voice, "We will suggest the most suitable tasks." This allows the reception desk to prioritize receiving relevant information by analyzing the user's social media activity. Some or all of the above processing in the reception desk may be performed using AI, for example, or not. For example, the reception desk can input the user's social media activity data into a generating AI and have the generating AI select relevant information.

[0083] The guidance unit can estimate the user's emotions and adjust the way the guidance is expressed based on the estimated emotions. For example, if the user is nervous, the guidance unit can provide guidance in a calm voice. For example, if the user is nervous, the guidance unit can say "Please calm down" and provide guidance in a calm voice. The guidance unit can also provide guidance in a cheerful voice if the user is relaxed. For example, if the user is relaxed, the guidance unit can say "Please relax and proceed with the work" and provide guidance in a cheerful voice. The guidance unit can also provide quick and concise guidance if the user is in a hurry. For example, if the user is in a hurry, the guidance unit can say "Please proceed with the work quickly" and provide concise guidance. In this way, by adjusting the way the guidance is expressed according to the user's emotions, guidance that is easy for the user to understand can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. Some or all of the above-described processes in the guidance unit may be performed using AI, for example, or without AI. For example, the guidance unit can input user emotion data into a generating AI and have the generating AI perform emotion estimation and adjustment of the guidance expression method.

[0084] The guidance unit can adjust the level of detail of the guidance based on the importance of the task when providing guidance. For example, if the task is important, the guidance unit will provide detailed guidance. For example, if the task is important, the guidance unit will announce in voice, "This task is important, so we will provide detailed guidance," and then provide detailed guidance. The guidance unit can also provide concise guidance for simple tasks. For example, if the task is simple, the guidance unit will announce in voice, "This task is simple, so we will provide concise guidance," and then provide concise guidance. The guidance unit can also adjust the level of detail of the guidance according to the importance of the task. For example, depending on the importance of the task, the guidance unit will announce in voice, "We will adjust the level of detail of the guidance according to the importance of the task," and then provide guidance with an appropriate level of detail. In this way, appropriate guidance can be provided by adjusting the level of detail of the guidance based on the importance of the task. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without using AI. For example, the guidance unit can input task importance data into a generating AI and have the generating AI perform the adjustment of the level of detail of the guidance.

[0085] The guidance unit can apply different guidance algorithms depending on the category of work when providing guidance. For example, in the case of customer service, the guidance unit provides customer service guidance. For example, in the case of customer service, the guidance unit will announce "We will provide customer service guidance" and then provide customer service guidance. The guidance unit can also provide data entry guidance in the case of clerical work. For example, in the case of clerical work, the guidance unit will announce "We will provide data entry guidance" and then provide data entry guidance. Furthermore, the guidance unit can apply the most suitable guidance algorithm depending on the category of work. For example, depending on the category of work, the guidance unit will announce "We will apply the most suitable guidance algorithm according to the category of work" and then provide appropriate guidance. This allows users to accurately perform their tasks by applying the most suitable guidance algorithm according to the category of work. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without using AI. For example, the guidance unit can input the category of work data into a generating AI and have the generating AI execute the application of the guidance algorithm.

[0086] The guidance unit can estimate the user's emotions and adjust the length of the guidance based on the estimated emotions. For example, if the user is in a hurry, the guidance unit can provide short, concise guidance. For instance, if the user is in a hurry, the guidance unit might announce, "We will now provide short, concise guidance," and then provide short, concise guidance. The guidance unit can also provide longer guidance with more detailed explanations if the user is relaxed. For example, if the user is relaxed, the guidance unit might announce, "We will now provide longer guidance with more detailed explanations," and then provide longer guidance with more detailed explanations. The guidance unit can also provide guidance with visually stimulating effects if the user is excited. For example, if the user is excited, the guidance unit might announce, "We will now provide guidance with visually stimulating effects," and then provide guidance with visually stimulating effects. By adjusting the length of the guidance according to the user's emotions, the guidance unit can provide guidance that is easy for the user to understand. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. The generative AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the processing described above in the guidance unit may be performed using AI, or not using AI. For example, the guidance unit may input user emotion data into the generative AI and have the generative AI perform emotion estimation and adjustment of the guidance length.

[0087] The guidance unit can determine the priority of guidance based on the start date of the work when providing guidance. For example, if the start date of the work is approaching, the guidance unit will provide guidance preferentially. For example, if the start date of the work is approaching, the guidance unit will announce via voice, "Since the start date of the work is approaching, we will provide guidance preferentially," and then provide the guidance preferentially. The guidance unit can also postpone providing guidance if the start date of the work is far away. For example, if the start date of the work is far away, the guidance unit will announce via voice, "Since the start date of the work is far away, we will provide guidance later," and then provide the guidance later. The guidance unit can also determine the priority of guidance based on the start date of the work. For example, based on the start date of the work, the guidance unit will announce via voice, "We will determine the priority of guidance based on the start date of the work," and then provide the guidance at the appropriate time. In this way, by determining the priority of guidance based on the start date of the work, guidance can be provided at the appropriate time. Some or all of the above processing in the guidance unit may be performed using AI, for example, or not using AI. For example, the guidance unit can input data on the start date of operations into the generating AI, and have the generating AI determine the priority of the guidance.

[0088] The guidance unit can adjust the order of guidance based on the relevance of the tasks when providing guidance. For example, the guidance unit can provide guidance in order of relevance, starting with the most relevant tasks. For example, the guidance unit can provide guidance by announcing, "We will provide guidance in order of relevance," starting with the most relevant tasks. The guidance unit can also postpone providing guidance for less relevant tasks. For example, the guidance unit can postpone less relevant tasks by announcing, "We will postpone providing guidance for less relevant tasks," and then provide guidance. The guidance unit can also adjust the order of guidance based on the relevance of the tasks. For example, the guidance unit can provide guidance in an appropriate order by announcing, "We will adjust the order of guidance based on the relevance of the tasks," based on the relevance of the tasks. This allows for efficient guidance provision by adjusting the order of guidance based on the relevance of the tasks. Some or all of the above processing in the guidance unit may be performed using AI, for example, or without AI. For example, the guidance unit can input task relevance data into a generating AI and have the generating AI perform the adjustment of the guidance order.

[0089] The display unit can estimate the user's emotions and adjust the display method based on the estimated emotions. For example, if the user is nervous, the display unit can provide a simple and highly visible display method. For example, if the user is nervous, the display unit can announce, "We will provide a simple and highly visible display method," and then provide a simple and highly visible display method. The display unit can also provide a display method that includes detailed information if the user is relaxed. For example, if the user is relaxed, the display unit can announce, "We will provide a display method that includes detailed information," and then provide a display method that includes detailed information. The display unit can also provide a display method that gets to the point if the user is in a hurry. For example, if the user is in a hurry, the display unit can announce, "We will provide a display method that gets to the point," and then provide a display method that gets to the point. In this way, by adjusting the display method according to the user's emotions, it is possible to provide a display that is easy for the user to understand. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. The generating AI may be a text generating AI (e.g., LLM) or a multimodal generating AI, but is not limited to such examples. Some or all of the processing described above in the display unit may be performed using AI, or not using AI. For example, the display unit can input user emotion data into the generating AI and have the generating AI perform emotion estimation and adjustment of the display method.

[0090] The display unit can adjust the level of detail displayed based on the importance of the work procedure. For example, if the work procedure is important, the display unit will provide a detailed display. For example, if the work procedure is important, the display unit will announce in voice, "This is an important work procedure, so we will provide a detailed display," and then provide a detailed display. The display unit can also provide a concise display for simple work procedures. For example, if the work procedure is simple, the display unit will announce in voice, "This is a simple work procedure, so we will provide a concise display," and then provide a concise display. The display unit can also adjust the level of detail displayed according to the importance of the work procedure. For example, depending on the importance of the work procedure, the display unit will announce in voice, "The level of detail displayed will be adjusted according to the importance of the work procedure," and then provide a display with an appropriate level of detail. In this way, by adjusting the level of detail displayed based on the importance of the work procedure, an appropriate display can be provided. Some or all of the above processing in the display unit may be performed using AI, for example, or without using AI. For example, the display unit can input importance data for business procedures into a generating AI, and have the generating AI adjust the level of detail displayed.

[0091] The display unit can apply different display algorithms depending on the category of the work procedure when displaying information. For example, in the case of customer service, the display unit can provide a display of customer service. For example, in the case of customer service, the display unit can announce "We will now provide a display of customer service" and then provide a display of customer service. The display unit can also provide a display of data entry in the case of office work. For example, in the case of office work, the display unit can announce "We will now provide a display of data entry" and then provide a display of data entry. Furthermore, the display unit can apply the most suitable display algorithm depending on the category of the work procedure. For example, depending on the category of the work procedure, the display unit can announce "We will now apply the most suitable display algorithm according to the category of the work procedure" and then provide an appropriate display. This allows the user to accurately perform their tasks by applying the most suitable display algorithm according to the category of the work procedure. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the category data of the work procedure into a generating AI and have the generating AI execute the application of the display algorithm.

[0092] The display unit can estimate the user's emotions and adjust the length of the display based on the estimated emotions. For example, if the user is in a hurry, the display unit can provide a short, concise display. For example, if the user is in a hurry, the display unit can announce, "We will now provide a short, concise display," and then provide a short, concise display. The display unit can also provide a longer display with detailed explanations if the user is relaxed. For example, if the user is relaxed, the display unit can announce, "We will now provide a longer display with detailed explanations," and then provide a longer display with detailed explanations. The display unit can also provide a display with visually stimulating effects if the user is excited. For example, if the user is excited, the display unit can announce, "We will now provide a display with visually stimulating effects," and then provide a display with visually stimulating effects. By adjusting the length of the display according to the user's emotions, the display can be made easier for the user to understand. Emotion estimation is achieved using an emotion estimation function, for example, with an emotion engine or generative AI. The generating AI may be, but is not limited to, text generating AI (e.g., LLM) or multimodal generating AI. Some or all of the above-described processing in the display unit may be performed using AI, or not using AI. For example, the display unit can input user emotion data into the generating AI and have the generating AI perform emotion estimation and adjustment of the display length.

[0093] The display unit can determine the display priority based on the start time of the business procedure. For example, if the start time of the business procedure is approaching, the display unit will prioritize displaying it. For example, if the start time of the business procedure is approaching, the display unit will announce by voice, "The start time of the business procedure is approaching, so we will display it first," and then display it first. The display unit can also postpone displaying the business procedure if the start time is far off. For example, if the start time of the business procedure is far off, the display unit will announce by voice, "The start time of the business procedure is far off, so we will display it first," and then display it first. The display unit can also determine the display priority based on the start time of the business procedure. For example, based on the start time of the business procedure, the display unit will announce by voice, "We will determine the display priority based on the start time of the business procedure," and then provide the display at the appropriate time. This allows the display to be provided at the appropriate time by determining the display priority based on the start time of the business procedure. Some or all of the above processing in the display unit may be performed using AI, for example, or without using AI. For example, the display unit can input data on the start time of business procedures into the generating AI, and have the generating AI determine the priority of the display.

[0094] The display unit can adjust the display order based on the relevance of the work procedures during display. For example, the display unit can display work procedures in order of relevance. For example, the display unit can provide a voice announcement saying, "We will now display the work procedures in order of relevance," and then display the work procedures in order of relevance. The display unit can also postpone the display of less relevant work procedures. For example, the display unit can postpone the display of less relevant work procedures and provide a voice announcement saying, "We will now postpone the display of less relevant work procedures." The display unit can also adjust the display order based on the relevance of the work procedures. For example, the display unit can provide a voice announcement saying, "We will adjust the display order based on the relevance of the work procedures," and then display the work procedures in the appropriate order. This allows for efficient display by adjusting the display order based on the relevance of the work procedures. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the relevance data of the work procedures into a generating AI and have the generating AI perform the adjustment of the display order.

[0095] The alert unit can estimate the user's emotions and adjust the way the alert is delivered based on the estimated emotions. For example, if the user is nervous, the alert unit can provide an alert in a calm voice. For instance, if the user is nervous, the alert unit can say "Please calm down" and deliver the alert in a calm voice. The alert unit can also provide an alert in a cheerful voice if the user is relaxed. For example, if the user is relaxed, the alert unit can say "Please relax and proceed with your work" and deliver the alert in a cheerful voice. The alert unit can also provide a quick and concise alert if the user is in a hurry. For example, if the user is in a hurry, the alert unit can say "Please proceed with your work quickly" and deliver the alert in a concise voice. In this way, by adjusting the way the alert is delivered according to the user's emotions, it is possible to provide alerts that are easy for the user to understand. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the processing described above in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input user emotion data into a generating AI and have the generating AI perform emotion estimation and adjust the way the alert is expressed.

[0096] The alert unit can adjust the level of detail of an alert based on the severity of the error when providing an alert. For example, in the case of a critical error, the alert unit provides a detailed alert. For example, in the case of a critical error, the alert unit will announce in voice, "Due to a critical error, a detailed alert will be provided," and then provide a detailed alert. The alert unit can also provide a concise alert in the case of a minor error. For example, in the case of a minor error, the alert unit will announce in voice, "Due to a minor error, a concise alert will be provided," and then provide a concise alert. The alert unit can also adjust the level of detail of an alert according to the severity of the error. For example, depending on the severity of the error, the alert unit will announce in voice, "The level of detail of the alert will be adjusted according to the severity of the error," and then provide an alert with the appropriate level of detail. In this way, by adjusting the level of detail of the alert based on the severity of the error, an appropriate alert can be provided. Some or all of the above processing in the alert unit may be performed using AI, for example, or without using AI. For example, the alert unit can input error severity data into a generating AI and have the generating AI perform the adjustment of the level of detail of the alert.

[0097] The alert unit can apply different alert algorithms depending on the error category when providing an alert. For example, in the case of a system error, the alert unit provides a technical alert. For example, in the case of a system error, the alert unit will announce, "Due to a system error, we will provide a technical alert," and then provide the technical alert. The alert unit can also provide an alert indicating how to perform the operation in the case of a user operation error. For example, in the case of a user operation error, the alert unit will announce, "Due to a user operation error, we will provide an alert indicating how to perform the operation," and then provide the alert indicating how to perform the operation. The alert unit can also apply the most suitable alert algorithm depending on the error category. For example, depending on the error category, the alert unit will announce, "We will apply the most suitable alert algorithm according to the error category," and then provide the appropriate alert. This allows users to quickly correct errors by applying the most suitable alert algorithm according to the error category. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input error category data into a generating AI and have the generating AI execute the application of the alert algorithm.

[0098] The alert unit can estimate the user's emotions and adjust the length of the alert based on the estimated emotions. For example, if the user is in a hurry, the alert unit can provide a short, to-the-point alert. For instance, if the user is in a hurry, the alert unit might announce, "We will now provide a short, to-the-point alert," and then provide a short, to-the-point alert. The alert unit can also provide a longer alert with more detailed explanations if the user is relaxed. For example, if the user is relaxed, the alert unit might announce, "We will now provide a longer alert with more detailed explanations," and then provide a longer alert with more detailed explanations. The alert unit can also provide an alert with visually stimulating effects if the user is excited. For example, if the user is excited, the alert unit might announce, "We will now provide an alert with visually stimulating effects," and then provide an alert with visually stimulating effects. By adjusting the length of the alert according to the user's emotions, the system can provide alerts that are easy for the user to understand. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the processing described above in the alert unit may be performed using AI, or not using AI. For example, the alert unit may input user sentiment data into the generation AI and have the generation AI perform sentiment estimation and adjust the length of the alert.

[0099] The alert unit can determine the priority of alerts based on when an error occurs. For example, if the timing of an error is imminent, the alert unit will prioritize providing the alert. For example, if the timing of an error is imminent, the alert unit will announce, "Since the timing of an error is imminent, we will prioritize providing the alert," and then provide the alert. The alert unit can also postpone providing the alert if the timing of an error is far off. For example, if the timing of an error is far off, the alert unit will announce, "Since the timing of an error is far off, we will postpone providing the alert," and then provide the alert. The alert unit can also determine the priority of alerts based on when the error occurs. For example, based on when the error occurs, the alert unit will announce, "We will determine the priority of alerts based on when the error occurred," and then provide the alert at the appropriate time. This allows for timely alert delivery by prioritizing alerts based on when the error occurs. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input error occurrence data into the generating AI and have the generating AI determine the priority of alerts.

[0100] The alert unit can adjust the order of alerts based on the relevance of the errors when providing alerts. For example, the alert unit can provide alerts in order of relevance, starting with the most relevant errors. For example, the alert unit can provide alerts in order of relevance, announcing, "We will provide alerts in order of relevance," and then provide the alerts. The alert unit can also postpone providing alerts for less relevant errors. For example, the alert unit can postpone providing alerts for less relevant errors, announcing, "We will postpone providing alerts for less relevant errors," and then provide the alerts. The alert unit can also adjust the order of alerts based on the relevance of the errors. For example, the alert unit can provide alerts in the appropriate order, announcing, "We will adjust the order of alerts based on the relevance of the errors," and then providing the alerts in the appropriate order. This allows for efficient alert delivery by adjusting the order of alerts based on the relevance of the errors. Some or all of the above processing in the alert unit may be performed using AI, for example, or without AI. For example, the alert unit can input error relevance data into a generating AI and have the generating AI perform the adjustment of the alert order.

[0101] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0102] The reception desk can analyze a user's past input history and select the most suitable reception method. For example, if a user has preferred using voice input in the past, it will prioritize suggesting voice input. The reception desk will then verbally ask, "Would you like to use voice input?" if the user has preferred using voice input in the past. It can also prioritize suggesting text input if the user has frequently used text input in the past. The reception desk will then verbally ask, "Would you like to use text input?" if the user has frequently used text input in the past. Furthermore, if a user has previously entered data during a specific time period, the reception desk can adjust the reception process to match that time period. The reception desk will then verbally ask, "Would you like to start entering data?" if the user has previously entered data during a specific time period. In this way, by analyzing a user's past input history, the system can provide the most suitable reception method.

[0103] The guidance unit can estimate the user's emotions and adjust the way it delivers guidance based on those emotions. For example, if the user is nervous, it can provide guidance in a calm voice. The guidance unit will say "Please calm down" and deliver guidance in a calm voice if the user is nervous. It can also provide guidance in a cheerful voice if the user is relaxed. The guidance unit will say "Please relax and proceed with the work" and deliver guidance in a cheerful voice if the user is relaxed. It can also provide quick and concise guidance if the user is in a hurry. The guidance unit will say "Please proceed with the work quickly" and deliver concise guidance if the user is in a hurry. In this way, by adjusting the way guidance is delivered according to the user's emotions, it is possible to provide guidance that is easy for the user to understand.

[0104] The display unit can estimate the user's emotions and adjust the display method based on those emotions. For example, if the user is nervous, it can provide a simple and highly visible display method. If the user is nervous, the display unit will announce, "We will provide a simple and highly visible display method," and then provide a simple and highly visible display method. If the user is relaxed, it can also provide a display method that includes detailed information. If the user is relaxed, the display unit will announce, "We will provide a display method that includes detailed information," and then provide a display method that includes detailed information. If the user is in a hurry, it can also provide a display method that focuses on the essentials. If the user is in a hurry, the display unit will announce, "We will provide a display method that focuses on the essentials," and then provide a display method that focuses on the essentials. In this way, by adjusting the display method according to the user's emotions, it is possible to provide a display that is easy for the user to understand.

[0105] The alert unit can estimate the user's emotions and adjust the way the alert is delivered based on those emotions. For example, if the user is nervous, the alert unit can provide an alert in a calm voice, saying "Please calm down." If the user is nervous, the alert unit can also provide an alert in a cheerful voice, saying "Please relax and proceed with your work." If the user is relaxed, the alert unit can also provide a quick and concise alert, saying "Please proceed with your work quickly." In this way, by adjusting the way the alert is delivered according to the user's emotions, it is possible to provide alerts that are easy for the user to understand.

[0106] The reception desk can estimate the user's emotions and prioritize the information to be received based on those emotions. For example, if the user is stressed, important information will be prioritized. The reception desk will announce to the user via voice, "We will prioritize receiving important information." If the user is relaxed, detailed information will also be prioritized. The reception desk will announce to the user via voice, "We will prioritize receiving detailed information." If the user is in a hurry, information that needs to be processed quickly will also be prioritized. The reception desk will announce to the user via voice, "We will prioritize receiving information that needs to be processed quickly." In this way, by prioritizing the information received according to the user's emotions, important information can be prioritized.

[0107] The guidance unit can adjust the level of detail in the guidance provided based on the importance of the task. For example, for important tasks, it provides detailed guidance. For important tasks, the guidance unit will announce, "This task is important, so we will provide detailed guidance," and then provide detailed guidance. For simple tasks, it can also provide concise guidance. For simple tasks, the guidance unit will announce, "This task is simple, so we will provide concise guidance," and then provide concise guidance. Furthermore, it can adjust the level of detail in the guidance according to the importance of the task. Depending on the importance of the task, the guidance unit will announce, "We will adjust the level of detail in the guidance according to the importance of the task," and then provide guidance with the appropriate level of detail. This allows for the provision of appropriate guidance by adjusting the level of detail based on the importance of the task.

[0108] The display unit can adjust the level of detail displayed based on the importance of the work procedure. For example, for important work procedures, it provides a detailed display. For important work procedures, the display unit will announce, "This is an important work procedure, so we will provide a detailed display," and then provide the detailed display. For simple work procedures, it can also provide a concise display. For simple work procedures, the display unit will announce, "This is a simple work procedure, so we will provide a concise display," and then provide the concise display. Furthermore, it can adjust the level of detail displayed according to the importance of the work procedure. Depending on the importance of the work procedure, the display unit will announce, "The level of detail displayed will be adjusted according to the importance of the work procedure," and then provide a display with an appropriate level of detail. In this way, by adjusting the level of detail displayed based on the importance of the work procedure, an appropriate display can be provided.

[0109] The alerting unit can adjust the level of detail of an alert based on the severity of the error when providing an alert. For example, in the case of a critical error, it provides a detailed alert. In the case of a critical error, the alerting unit will announce, "Due to a critical error, a detailed alert will be provided," and then provide a detailed alert. It can also provide a concise alert in the case of a minor error. In the case of a minor error, the alerting unit will announce, "Due to a minor error, a concise alert will be provided," and then provide a concise alert. Furthermore, it can adjust the level of detail of an alert according to the severity of the error. Depending on the severity of the error, the alerting unit will announce, "The level of detail of the alert will be adjusted according to the severity of the error," and then provide an alert with the appropriate level of detail. In this way, by adjusting the level of detail of the alert based on the severity of the error, appropriate alerts can be provided.

[0110] The reception desk can prioritize receiving highly relevant information by considering the user's geographical location during the input process. For example, if a user is in a specific region, it will prioritize receiving tasks related to that region. The reception desk will announce to the user via voice, "We will prioritize receiving tasks related to that region." Furthermore, if a user is on the move, it can prioritize receiving tasks related to their current location. The reception desk will announce to the user via voice, "We will prioritize receiving tasks related to your current location." Additionally, if a user is in a specific location, it can prioritize receiving information related to that location. The reception desk will announce to the user via voice, "We will prioritize receiving information related to that location." This allows the reception desk to prioritize receiving highly relevant information by considering the user's geographical location.

[0111] The guidance unit can prioritize guidance based on the start date of the work when providing guidance. For example, if the start date of the work is approaching, guidance will be provided preferentially. If the start date of the work is approaching, the guidance unit will announce via voice, "Since the start date of the work is approaching, we will provide guidance preferentially," and then provide the guidance preferentially. Conversely, if the start date of the work is far off, guidance can be postponed. If the start date of the work is far off, the guidance unit will announce via voice, "Since the start date of the work is far off, we will provide guidance later," and then provide the guidance later. The guidance unit can also determine the priority of guidance based on the start date of the work. Based on the start date of the work, the guidance unit will announce via voice, "We will determine the priority of guidance based on the start date of the work," and provide guidance at the appropriate time. This ensures that guidance is provided at the appropriate time by determining the priority of guidance based on the start date of the work.

[0112] The following briefly describes the processing flow for example form 2.

[0113] Step 1: The reception desk receives user input. User input includes voice input, touch input, and keyboard input. For example, the reception desk uses voice recognition technology to analyze the user's voice input and receive instructions. Touch input is a method in which the user enters instructions by operating a touch panel, allowing for intuitive operation. Keyboard input is a method in which the user enters instructions using a keyboard, allowing for accurate input. Step 2: The guidance unit provides voice guidance based on the information received by the reception unit. The voice guidance is provided based on the type of voice, the content of the guidance, and the timing of its provision. For example, the guidance unit explains the basic procedures of the work to the user via voice. It also provides information necessary for the user to start the work via voice, and if the user makes a mistake in the work procedure, it guides them to the correct procedure via voice. Step 3: The display unit displays the work procedure in AR based on the audio guidance provided by the guidance unit. The AR display is performed using a smartphone or AR glasses. For example, the display unit can display the work procedure on a smartphone screen, and the work procedure can be visually shown using AR glasses. Each step of the work procedure is shown in detail and displayed in a visually easy-to-understand manner for the user. Step 4: The alert unit provides error prevention and mitigation alerts based on the work procedures displayed by the display unit. Error prevention and mitigation alerts are provided based on the type of error, the content of the alert, and the timing of its provision. For example, the alert unit provides an alert if the user makes a mistake in the work procedure, and provides specific corrective methods when an error occurs. It can also provide advance warnings to prevent errors.

[0114] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0115] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0116] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0117] Each of the multiple elements described above, including the reception unit, guidance unit, display unit, and alert unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the smart device 14 and accepts voice and touch input from the user. The guidance unit is implemented by the specific processing unit 290 of the data processing unit 12 and provides voice guidance. The display unit displays business procedures using AR on the display 40A of the smart device 14, for example. The alert unit is implemented by the specific processing unit 290 of the data processing unit 12 and provides error prevention and prevention alerts. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0118] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0119] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0120] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0121] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0122] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0123] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0124] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0125] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0126] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0127] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0128] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0129] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0130] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0131] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0132] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0133] Each of the multiple elements described above, including the reception unit, guidance unit, display unit, and alert unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the smart glasses 214 and accepts voice and touch input from the user. The guidance unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and provides voice guidance. The display unit displays business procedures in augmented reality using the display of the smart glasses 214. The alert unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and provides error prevention and prevention alerts. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0134] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0135] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0136] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0137] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0138] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0139] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0140] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0141] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0142] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0143] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0144] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0145] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0146] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0147] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0148] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0149] Each of the multiple elements described above, including the reception unit, guidance unit, display unit, and alert unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the headset terminal 314 and accepts voice and touch input from the user. The guidance unit is implemented by the specific processing unit 290 of the data processing unit 12 and provides voice guidance. The display unit displays business procedures in augmented reality using the display 343 of the headset terminal 314. The alert unit is implemented by the specific processing unit 290 of the data processing unit 12 and provides error prevention and prevention alerts. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0150] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0151] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0152] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0153] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0154] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0155] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0156] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0157] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0158] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0159] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0160] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0161] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0162] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0163] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0164] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0165] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0166] Each of the multiple elements described above, including the reception unit, guidance unit, display unit, and alert unit, is implemented in at least one of the robot 414 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the robot 414 and accepts voice and touch input from the user. The guidance unit is implemented by the specific processing unit 290 of the data processing unit 12 and provides voice guidance. The display unit displays the work procedure using the display of the robot 414 in augmented reality. The alert unit is implemented by the specific processing unit 290 of the data processing unit 12 and provides error prevention and prevention alerts. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0167] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0168] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0169] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0170] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0171] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0172] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0173] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0174] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0175] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0176] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0177] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0178] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0179] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0180] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0181] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0182] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0183] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0184] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0185] (Note 1) A reception area that receives user input, A guidance unit provides voice guidance based on the information received by the reception unit, A display unit that displays the work procedure in AR based on the voice guidance provided by the aforementioned guidance unit, The system includes an alert unit that provides error prevention and prevention alerts based on the work procedures displayed by the aforementioned display unit. A system characterized by the following features. (Note 2) The aforementioned guidance unit, The system provides basic customer service procedures via voice guidance. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned display unit is Use smartphones or AR glasses to visually demonstrate work procedures. The system described in Appendix 1, characterized by the features described herein. (Note 4) The alert unit is, Provides alerts when a user makes a mistake in the work procedure. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned guidance unit, Provides users with the information they need to start their work. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned display unit is This document details each step of the work procedure. The system described in Appendix 1, characterized by the features described herein. (Note 7) The alert unit is, Provide specific instructions on how to fix errors when they occur. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned reception unit is The system estimates the user's emotions and adjusts the timing of input acceptance based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned reception unit is Analyze the user's past input history to select the optimal reception method. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned reception unit is When receiving input, filtering is performed based on the user's current situation and areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned reception unit is It estimates the user's emotions and determines the priority of the information to be received based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned reception unit is When receiving input, the system prioritizes receiving information that is highly relevant, taking into account the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned reception unit is When receiving input, the system analyzes the user's social media activity and collects relevant information. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned guidance unit, It estimates the user's emotions and adjusts the way guidance is presented based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned guidance unit, When providing guidance, adjust the level of detail based on the importance of the task. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned guidance unit, When providing guidance, different guidance algorithms are applied depending on the category of the task. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned guidance unit, It estimates the user's emotions and adjusts the length of the guidance based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned guidance unit, When providing guidance, prioritize the guidance based on the start date of operations. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned guidance unit, When providing guidance, adjust the order of the guidance based on the relevance of the task. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned display unit is It estimates the user's emotions and adjusts the display method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned display unit is When displaying, adjust the level of detail based on the importance of the work procedure. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned display unit is When displaying, different display algorithms are applied depending on the category of the business procedure. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned display unit is It estimates the user's emotions and adjusts the display length based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned display unit is When displaying information, the display priority is determined based on the start date of the business procedure. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned display unit is When displaying, the display order is adjusted based on the relevance of the work procedures. The system described in Appendix 1, characterized by the features described herein. (Note 26) The alert unit is, It estimates the user's emotions and adjusts how alerts are presented based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The alert unit is, When providing alerts, adjust the level of detail of the alert based on the severity of the error. The system described in Appendix 1, characterized by the features described herein. (Note 28) The alert unit is, When providing alerts, different alert algorithms are applied depending on the error category. The system described in Appendix 1, characterized by the features described herein. (Note 29) The alert unit is, It estimates the user's sentiment and adjusts the length of the alert based on the estimated sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 30) The alert unit is, When providing alerts, prioritize the alerts based on when the error occurred. The system described in Appendix 1, characterized by the features described herein. (Note 31) The alert unit is, When providing alerts, the order of alerts will be adjusted based on the relevance of the errors. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0186] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A reception area that receives user input, A guidance unit provides voice guidance based on the information received by the reception unit, A display unit that displays the work procedure in AR based on the voice guidance provided by the aforementioned guidance unit, The system includes an alert unit that provides error prevention and prevention alerts based on the work procedures displayed by the aforementioned display unit. A system characterized by the following features.

2. The aforementioned guidance unit, The system provides basic customer service procedures via voice guidance. The system according to feature 1.

3. The aforementioned display unit is Use smartphones or AR glasses to visually demonstrate work procedures. The system according to feature 1.

4. The alert unit is, Provides alerts when a user makes a mistake in the work procedure. The system according to feature 1.

5. The aforementioned guidance unit, Provides users with the information they need to start their work. The system according to feature 1.

6. The aforementioned display unit is This document details each step of the work procedure. The system according to feature 1.

7. The alert unit is, Provide specific instructions on how to fix errors when they occur. The system according to feature 1.

8. The aforementioned reception unit is The system estimates the user's emotions and adjusts the timing of input acceptance based on the estimated emotions. The system according to feature 1.

Images