Artificial intelligence device and operation method thereof

Abstract

An artificial intelligence device according to an embodiment of the present disclosure may: receive a speech command uttered by a user; acquire an analysis result of the received speech command; transmit an additional health questionnaire request to an electronic device when the acquired analysis result is a request for the health diagnosis of the user; receive health questionnaire response information responding to the additional health questionnaire request from the electronic device; generate a prompt requesting disease diagnosis on the basis of health management information and the health questionnaire response information; transmit the generated prompt to a generative AI server; and receive a diagnosis result from the generative AI server.

Description

Artificial intelligence device and its operating method

[0001] The present disclosure relates to an artificial intelligence device, and more particularly, to an artificial intelligence device capable of diagnosing a user's health.

[0002] Concerns about health are growing, especially as the proportion of the elderly population increases and the number of single-person households increases.

[0003] In particular, most people regularly take care of their health, such as taking medication, managing their diet, exercising, and getting regular checkups at the hospital.

[0004] Additionally, there is a growing trend of various health programs, such as home training apps for health management through TV or smartphones.

[0005] Traditionally, biometric data collected using biometric measuring devices has been analyzed through a medical AI engine to provide medical services such as health checkups and disease prediction.

[0006] However, since health checkups / disease predictions are performed using only biometric data in the past and do not take into account the user's personal health information, there is a problem in that accurate health checkups and disease predictions for users are impossible.

[0007] The purpose of this disclosure is to generate prompts to accurately identify a user's disease that is input into a generative AI model.

[0008] The purpose of this disclosure is to automatically provide a hospital reservation service based on the diagnosis results output by a generative AI model.

[0009] An artificial intelligence device according to an embodiment of the present disclosure may include a memory that stores health management information of a user; a communication unit that communicates with an electronic device or a generation artificial intelligence (AI) server; and a processor that receives a voice command uttered by a user from the electronic device, obtains an analysis result of the received voice command, and if the obtained analysis result is a request for a health diagnosis of the user, transmits an additional questionnaire request to the electronic device through the communication unit, receives questionnaire response information in response to the additional questionnaire request from the electronic device through the communication unit, generates a prompt requesting a disease diagnosis based on the health management information and the questionnaire response information, transmits the generated prompt to the generation AI server through the communication unit, and receives a diagnosis result from the generation AI server through the communication unit.

[0010] An operating method of an artificial intelligence device according to an embodiment of the present disclosure may include the steps of: receiving a voice command uttered by a user from an electronic device; obtaining an analysis result of the received voice command; transmitting an additional questionnaire request to the electronic device when the obtained analysis result is a request for a health diagnosis of the user; receiving questionnaire response information responsive to the additional questionnaire request from the electronic device; generating a prompt requesting a disease diagnosis based on health management information and the questionnaire response information; transmitting the generated prompt to a generation AI server; and receiving a diagnosis result from the generation AI server.

[0011] According to embodiments of the present disclosure, accurate diagnostic results for the user's symptoms can be provided to the user by generating prompts using questionnaire response information, such as additional utterances, responses to a questionnaire, or the results of the user performing a specific action.

[0012] In addition, according to an embodiment of the present disclosure, a reservation service is provided that automatically reserves a hospital for treatment of the user's disease, thereby greatly improving the convenience of the user's health management.

[0013] Figure 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0014] Figure 2 is a block diagram of a remote control device according to one embodiment of the present invention.

[0015] Figure 3 shows an example of an actual configuration of a remote control device according to one embodiment of the present invention.

[0016] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0017] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0018] FIG. 6 is a ladder diagram for explaining an operation method of an AI system according to an embodiment of the present disclosure.

[0019] FIG. 7 is a drawing showing an example of a questionnaire according to one embodiment of the present disclosure.

[0020] FIG. 8 is a drawing illustrating a process of identifying a user's symptoms and providing a remote diagnosis system according to an embodiment of the present disclosure.

[0021] FIG. 9 is a diagram illustrating a process of collecting user health management information according to one embodiment of the present disclosure.

[0022] FIG. 10 is a diagram showing an example of outputting a generation response result according to one embodiment of the present disclosure.

[0023] FIG. 11A and FIG. 11B are diagrams illustrating an example of providing a hospital reservation service for treatment of a disease included in a generated response result according to one embodiment of the present disclosure.

[0024] Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0025] A display device according to an embodiment of the present invention is, for example, an intelligent display device that adds computer-assisted functionality to its broadcast reception function. While faithfully performing the broadcast reception function, it can also be equipped with Internet functionality and other features, providing a more user-friendly interface, such as a manual input device, touch screen, or space remote control. Furthermore, with support for wired or wireless Internet functionality, it can connect to the Internet and a computer, enabling functions such as email, web browsing, banking, or gaming. A standardized, general-purpose operating system can be used for these various functions.

[0026] Accordingly, the display device described in the present invention can perform various user-friendly functions, for example, by allowing various applications to be freely added or deleted on a general-purpose operating system kernel. More specifically, the display device can be a network TV, HBB TV, smart TV, LED TV, OLED TV, etc., and in some cases, it can also be applied to smartphones.

[0027] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0028] Referring to FIG. 1, the display device (100) may include a broadcast receiving unit (130), an external device interface (135), a memory (140), a user input interface (150), a controller (170), a wireless communication interface (173), a display (180), a speaker (185), and a power supply circuit (190).

[0029] The broadcast receiving unit (130) may include a tuner (131), a demodulator (132), and a network interface (133).

[0030] The tuner (131) can select a specific broadcast channel according to a channel selection command. The tuner (131) can receive a broadcast signal for the selected specific broadcast channel.

[0031] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0032] The external device interface (135) can receive an application or a list of applications within an adjacent external device and transmit it to the controller (170) or memory (140).

[0033] The external device interface (135) can provide a connection path between the display device (100) and the external device. The external device interface (135) can receive one or more of images and audio output from an external device connected wirelessly or wiredly to the display device (100) and transmit them to the controller (170). The external device interface (135) can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0034] A video signal of an external device input through an external device interface (135) can be output through a display (180). A voice signal of an external device input through an external device interface (135) can be output through a speaker (185).

[0035] An external device that can be connected to the external device interface (135) may be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0036] The network interface (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet. The network interface (133) may transmit or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0037] Additionally, some content data stored in the display device (100) can be transmitted to a selected user or electronic device among other users or other electronic devices pre-registered in the display device (100).

[0038] The network interface (133) can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with the corresponding server.

[0039] In addition, the network interface (133) can receive content or data provided by a content provider or network operator. That is, the network interface (133) can receive content such as movies, advertisements, games, VOD, broadcast signals, etc. and information related thereto provided from a content provider or network provider via a network.

[0040] Additionally, the network interface (133) can receive firmware update information and update files provided by the network operator, and transmit data to the Internet or content provider or network operator.

[0041] The network interface (133) can select and receive a desired application from among applications open to the public via a network.

[0042] The memory (140) stores a program for each signal processing and control within the controller (170), and can store signal-processed image, voice, or data signals.

[0043] In addition, the memory (140) may perform a function for temporary storage of video, audio, or data signals input from an external device interface (135) or a network interface (133), and may store information about a specific image through a channel memory function.

[0044] The memory (140) can store an application or a list of applications input from an external device interface (135) or a network interface (133).

[0045] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the memory (140) and provide them to the user.

[0046] The user input interface (150) can transmit a signal input by the user to the controller (170) or transmit a signal from the controller (170) to the user. For example, the user input interface (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF) communication, or infrared (IR) communication, or process the control signals from the controller (170) to be transmitted to the remote control device (200).

[0047] In addition, the user input interface (150) can transmit control signals input from local keys (not shown) such as a power key, channel key, volume key, and setting value to the controller (170).

[0048] An image signal processed by the controller (170) can be input to the display (180) and displayed as an image corresponding to the image signal. In addition, an image signal processed by the controller (170) can be input to an external output device through an external device interface (135).

[0049] The voice signal processed by the controller (170) can be output as audio to the speaker (185). In addition, the voice signal processed by the controller (170) can be input to an external output device through the external device interface (135).

[0050] In addition, the controller (170) can control the overall operation within the display device (100).

[0051] In addition, the controller (170) can control the display device (100) by a user command or internal program input through the user input interface (150), and can connect to a network to enable the user to download a desired application or application list into the display device (100).

[0052] The controller (170) enables the user-selected channel information, etc. to be output through a display (180) or speaker (185) together with processed video or audio signals.

[0053] In addition, the controller (170) allows a video signal or audio signal from an external device, for example, a camera or camcorder, input through the external device interface (135) to be output through the display (180) or speaker (185) in accordance with an external device video playback command received through the user input interface (150).

[0054] Meanwhile, the controller (170) can control the display (180) to display an image, for example, a broadcast image input through a tuner (131), an external input image input through an external device interface (135), an image input through a network interface, or an image stored in a memory (140) can be controlled to be displayed on the display (180). In this case, the image displayed on the display (180) can be a still image or a moving image, and can be a 2D image or a 3D image.

[0055] In addition, the controller (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from outside, and the content can be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0056] The wireless communication interface (173) can communicate with an external device through wired or wireless communication. The wireless communication interface (173) can perform short-range communication with the external device. To this end, the wireless communication interface (173) can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication interface (173) can support wireless communication between the display device (100) and a wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and a network in which the display device (100, or an external server) is located via a short-range wireless communication network (Wireless Area Network). The short-range wireless communication network can be a short-range wireless personal area network (Wireless Personal Area Network).

[0057] Here, the other display device (100) may be a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display (HMD)) or a mobile terminal such as a smart phone that can exchange data with (or be linked to) the display device (100) according to the present invention. The wireless communication interface (173) may detect (or recognize) a wearable device capable of communication around the display device (100).

[0058] Furthermore, if the detected wearable device is a device certified to communicate with the display device (100) according to the present invention, the controller (170) can transmit at least a portion of the data processed in the display device (100) to the wearable device via the wireless communication interface (173). Accordingly, a user of the wearable device can utilize the data processed in the display device (100) via the wearable device.

[0059] The display (180) can generate a driving signal by converting a video signal, data signal, OSD signal processed by the controller (170) or a video signal, data signal, etc. received from an external device interface (135) into R, G, and B signals, respectively.

[0060] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an embodiment of the present invention, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0061] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present invention, and their specific operations or devices do not limit the scope of the present invention.

[0062] According to another embodiment of the present invention, the display device (100) may receive and play back an image through a network interface (133) or an external device interface (135) without having a tuner (131) and a demodulator (132), unlike that shown in FIG. 1.

[0063] For example, the display device (100) may be implemented separately as an image processing device, such as a set-top box, for receiving broadcast signals or contents according to various network services, and a content playback device for playing contents input from the image processing device.

[0064] In this case, the operating method of the display device according to the embodiment of the present invention described below may be performed by any one of the display device (100) described with reference to FIG. 1, as well as an image processing device such as the separated set-top box, or a content playback device having a display (180) and a speaker (185).

[0065] Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3.

[0066] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present invention, and FIG. 3 shows an example of an actual configuration of a remote control device (200) according to an embodiment of the present invention.

[0067] First, referring to FIG. 2, the remote control device (200) may include a fingerprint recognition device (210), a wireless communication circuit (220), a user input interface (230), a sensor (240), an output interface (250), a power supply circuit (260), a memory (270), a controller (280), and a microphone (290).

[0068] Referring to FIG. 2, the wireless communication circuit (220) transmits and receives signals with any one of the display devices according to the embodiments of the present invention described above.

[0069] The remote control device (200) may be equipped with an RF circuit (221) capable of transmitting and receiving signals with the display device (100) according to RF communication standards, and an IR circuit (223) capable of transmitting and receiving signals with the display device (100) according to IR communication standards. In addition, the remote control device (200) may be equipped with a Bluetooth circuit (225) capable of transmitting and receiving signals with the display device (100) according to Bluetooth communication standards. In addition, the remote control device (200) may be equipped with an NFC circuit (227) capable of transmitting and receiving signals with the display device (100) according to NFC (Near Field Communication) communication standards, and a WLAN circuit (229) capable of transmitting and receiving signals with the display device (100) according to WLAN (Wireless LAN) communication standards.

[0070] Additionally, the remote control device (200) transmits a signal containing information about the movement of the remote control device (200) to the display device (100) through a wireless communication circuit (220).

[0071] Meanwhile, the remote control device (200) can receive a signal transmitted by the display device (100) through the RF circuit (221), and, if necessary, can transmit commands for turning the power on / off, changing the channel, changing the volume, etc. to the display device (100) through the IR circuit (223).

[0072] The user input interface (230) may be configured as a keypad, buttons, a touchpad, or a touch screen. The user can input commands related to the display device (100) to the remote control device (200) by operating the user input interface (230). If the user input interface (230) includes a hard key button, the user can input commands related to the display device (100) to the remote control device (200) by pushing the hard key button. This will be described with reference to FIG. 3.

[0073] Referring to FIG. 3, the remote control device (200) may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button (212), a power button (231), a home button (232), a live button (233), an external input button (234), a volume control button (235), a voice recognition button (236), a channel change button (237), a confirmation button (238), and a back button (239).

[0074] The fingerprint recognition button (212) may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button (212) may be capable of a push operation, and may receive a push operation and a fingerprint recognition operation.

[0075] The power button (231) may be a button for turning the power of the display device (100) on / off.

[0076] The home button (232) may be a button for moving to the home screen of the display device (100).

[0077] The live button (233) may be a button for displaying a real-time broadcast program.

[0078] The external input button (234) may be a button for receiving an external input connected to the display device (100).

[0079] The volume control button (235) may be a button for adjusting the size of the volume output by the display device (100).

[0080] The voice recognition button (236) may be a button for receiving a user's voice and recognizing the received voice.

[0081] The channel change button (237) may be a button for receiving a broadcast signal of a specific broadcast channel.

[0082] The confirmation button (238) may be a button for selecting a specific function, and the back button (239) may be a button for returning to the previous screen.

[0083] Let's explain Figure 2 again.

[0084] When the user input interface (230) has a touch screen, the user can input commands related to the display device (100) using the remote control device (200) by touching the soft keys of the touch screen. In addition, the user input interface (230) may have various types of input means that can be operated by the user, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

[0085] The sensor (240) may include a gyro sensor (241) or an acceleration sensor (243), and the gyro sensor (241) may sense information about the movement of the remote control device (200).

[0086] For example, the gyro sensor (241) can sense information about the operation of the remote control device (200) based on the x, y, and z axes, and the acceleration sensor (243) can sense information about the movement speed of the remote control device (200). Meanwhile, the remote control device (200) can further include a distance measuring sensor, so as to sense the distance to the display (180) of the display device (100).

[0087] The output interface (250) can output a video or audio signal corresponding to an operation of the user input interface (230) or a signal transmitted from the display device (100).

[0088] The user can recognize whether the output interface (250) is manipulating the user input interface (230) or controlling the display device (100).

[0089] For example, the output interface (250) may include an LED (251) that lights up when the user input interface (230) is operated or a signal is transmitted and received with the display device (100) via the wireless communication unit (225), a vibrator (253) that generates vibrations, a speaker (255) that outputs sound, or a display (257) that outputs images.

[0090] In addition, the power supply circuit (260) supplies power to the remote control device (200), and power waste can be reduced by stopping the power supply when the remote control device (200) does not move for a predetermined period of time.

[0091] The power supply circuit (260) can resume power supply when a predetermined key provided in the remote control device (200) is operated.

[0092] The memory (270) can store various types of programs, application data, etc. required for the control or operation of the remote control device (200).

[0093] When the remote control device (200) wirelessly transmits and receives signals through the display device (100) and the RF circuit (221), the remote control device (200) and the display device (100) transmit and receive signals through a predetermined frequency band.

[0094] The controller (280) of the remote control device (200) can store and reference information regarding the frequency band that can wirelessly transmit and receive signals with the display device (100) paired with the remote control device (200) in the memory (270).

[0095] The controller (280) controls all matters related to the control of the remote control device (200). The controller (280) can transmit a signal corresponding to a predetermined key operation of the user input interface (230) or a signal corresponding to the movement of the remote control device (200) sensed by the sensor (240) to the display device (100) via the wireless communication unit (225).

[0096] Additionally, the microphone (290) of the remote control device (200) can acquire voice.

[0097] A plurality of microphones (290) may be provided.

[0098] Next, Figure 4 is described.

[0099] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0100] Figure 4 (a) illustrates that a pointer (205) corresponding to a remote control device (200) is displayed on a display (180).

[0101] The user can move or rotate the remote control device (200) up and down, left and right. The pointer (205) displayed on the display (180) of the display device (100) corresponds to the movement of the remote control device (200). This remote control device (200) can be called a space remote control because, as shown in the drawing, the pointer (205) moves and is displayed according to the movement in 3D space.

[0102] Figure 4 (b) illustrates that when a user moves the remote control device (200) to the left, the pointer (205) displayed on the display (180) of the display device (100) also moves to the left correspondingly.

[0103] Information about the movement of the remote control device (200) detected through the sensor of the remote control device (200) is transmitted to the display device (100). The display device (100) can calculate the coordinates of the pointer (205) from the information about the movement of the remote control device (200). The display device (100) can display the pointer (205) to correspond to the calculated coordinates.

[0104] Figure 4 (c) illustrates a case where a user moves the remote control device (200) away from the display (180) while pressing a specific button within the remote control device (200). As a result, a selection area within the display (180) corresponding to the pointer (205) can be zoomed in and displayed in an enlarged manner.

[0105] Conversely, when the user moves the remote control device (200) closer to the display (180), the selection area within the display (180) corresponding to the pointer (205) may be zoomed out and displayed in a reduced size.

[0106] Meanwhile, when the remote control device (200) moves away from the display (180), the selection area may be zoomed out, and when the remote control device (200) moves closer to the display (180), the selection area may be zoomed in.

[0107] In addition, when a specific button within the remote control device (200) is pressed, recognition of up, down, left, and right movements may be excluded. That is, when the remote control device (200) moves away from or toward the display (180), up, down, left, and right movements may not be recognized, and only forward and backward movements may be recognized. When a specific button within the remote control device (200) is not pressed, only the pointer (205) moves in accordance with the up, down, left, and right movements of the remote control device (200).

[0108] Meanwhile, the movement speed or movement direction of the pointer (205) can correspond to the movement speed or movement direction of the remote control device (200).

[0109] Meanwhile, the pointer in this specification refers to an object displayed on the display (180) in response to the operation of the remote control device (200). Accordingly, objects of various shapes other than the arrow shape illustrated in the drawing can be used as the pointer (205). For example, the pointer may be a concept including a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer (205) may be displayed corresponding to one point on the horizontal or vertical axis on the display (180), or may be displayed corresponding to multiple points such as lines or surfaces.

[0110] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0111] Referring to FIG. 5, the AI ​​server (500) may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network.

[0112] The AI ​​server (500) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network.

[0113] The AI ​​server (500) may be included as part of the configuration of the AI ​​device (100) and may perform at least part of the AI ​​processing together.

[0114] The AI ​​server (500) may include a communication unit (510), a memory (530), a learning processor (540), and a processor (560).

[0115] The communication unit (510) can transmit and receive data with an external device such as a display device (100).

[0116] The memory (530) may include a model storage unit (531).

[0117] The model storage unit (531) can store a model (or artificial neural network, 531a) that is being learned or has been learned through the learning processor (540).

[0118] A learning processor (540) can train an artificial neural network (531a) using learning data. The learning model can be used while mounted on the AI ​​server (500) of the artificial neural network, or can be mounted on an external device such as a display device (100).

[0119] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (530).

[0120] The processor (560) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0121] FIG. 6 is a ladder diagram for explaining an operation method of an AI system according to an embodiment of the present disclosure.

[0122] Referring to FIG. 6, the AI ​​system may include a display device (100), an AI server (500), and a generation AI server (610).

[0123] The AI ​​server (500) may be a natural language processing (NLP) server.

[0124] The AI ​​server (500) may be referred to as an artificial intelligence device.

[0125] The controller (170) of the display device (100) can receive voice commands (S601).

[0126] In one embodiment, the controller (170) can receive a voice command spoken by a user through a microphone (not shown).

[0127] The controller (170) can receive a voice command from a remote control device (200). The remote control device (200) can receive a voice command spoken by a user and transmit it to the display device (100).

[0128] The controller (170) of the display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) via the network interface (133) (S603).

[0129] In one embodiment, the controller (170) may transmit voice data along with user identification information to the AI ​​server (500) via a network interface (133).

[0130] The processor (560) of the AI ​​server (500) can obtain the analysis result of the voice command based on the received voice data (S605).

[0131] In one embodiment, the processor (560) can convert voice data into text data using a speech to text (STT) engine stored in the memory (530).

[0132] In another embodiment, the processor (560) can transmit voice data to an STT server and receive converted text data from the STT server.

[0133] The processor (560) can obtain analysis results of text data by using a natural language processing engine (NLP) stored in the memory (530) for the converted text data.

[0134] The analysis results may include analysis results on the intent of the voice command.

[0135] For example, voice commands In this case, the analysis result may indicate an intention to diagnose the user's health. That is, the analysis result of the voice command may indicate an intention to request a diagnosis of a disease or the presence or absence of a disease.

[0136] The processor (560) of the AI ​​server (500) can determine whether additional questioning is necessary based on the analysis result of the voice command (S607).

[0137] In one embodiment, the processor (560) may determine that additional questioning is necessary if the user's symptoms are not included in the analysis results.

[0138] In another embodiment, the processor (560) may determine that additional questioning is necessary even if the analysis results include the user's symptoms, if additional diagnostic information is needed to determine the diagnosis.

[0139] If the processor (560) of the AI ​​server (500) determines that additional questioning is necessary, it can transmit an additional questioning request to the display device (100) through the communication unit (510) (S609).

[0140] The additional questionnaire request may include one or more of a questionnaire, a request for additional utterances from the user, or a request for results for performing a specific action.

[0141] In one embodiment, the additional questionnaire request may include a questionnaire previously stored in a health management database. The health management database may be the memory (530) of the AI ​​server (500) or may be provided separately from the AI ​​server (500).

[0142] In one embodiment, the questionnaire may be obtained from the generation AI server (600). If the processor (560) determines that additional questionnaires are necessary, it may request the questionnaire from the generation AI server (600) and receive the questionnaire from the generation AI server (600).

[0143] If the processor (560) has already stored answers to some of the multiple questions included in the questionnaire, it may include only the remaining questions in the questionnaire.

[0144] In another embodiment, the additional questionnaire request may be a request to inquire about one or more of the types of symptoms or the areas where the symptoms are present.

[0145] The processor (560) may receive the user's response to the initial questionnaire before step S601.

[0146] If the processor (560) determines that additional questionnaires are necessary based on the received user response, it can generate an additional questionnaire request and transmit it to the display device (100).

[0147] The additional questionnaire request may include one or more of a questionnaire, a request for additional utterances from the user, or a request for results for performing a specific action.

[0148] The communication unit (510) may be referred to as a communication interface. The communication unit (510) may include a circuit for Internet communication.

[0149] FIG. 7 is a drawing showing an example of a questionnaire according to one embodiment of the present disclosure.

[0150] The AI ​​server (500) can transmit the questionnaire (700) to the display device (100). The questionnaire (700) may be generated by the generation AI server (600).

[0151] The questionnaire (700) may be a form that inquires about the user's personal information, existing diseases, allergies, surgical history, types of symptoms, areas of symptoms, etc.

[0152] The display device (100) can receive a response to an inquiry in the questionnaire (700).

[0153] Again, Figure 6 is explained.

[0154] The controller (170) of the display device (100) can obtain questionnaire response information in response to an additional questionnaire request and transmit the obtained questionnaire response information to the AI ​​server (500) via the network interface (133) (S611).

[0155] In one embodiment, the questionnaire response information may include a response to a questionnaire. In this case, the controller (170) may receive the response to the questionnaire through text input or speech input.

[0156] In another embodiment, if the additional questionnaire request is a request to inquire about one or more of the types of symptoms or the areas where the symptoms are present, the controller (170) may output a notification to input the types of symptoms, the areas where the symptoms are present, or the results of performing a specific action.

[0157] The controller (170) can receive one or more types of symptoms or areas where symptoms are present through the user input interface (150). The controller (170) can receive information on additional voice commands additionally uttered by the user or the results of a specific action performed by the user, and transmit information on the received additional voice commands or specific actions to the AI ​​server (500) through the network interface (133).

[0158] For example, information about the result of performing a specific action may be one or more of a blood sugar measurement value or a blood pressure measurement value.

[0159] The processor (560) of the AI ​​server (500) can generate a prompt for a response from the generated AI based on the user's health management information and questionnaire response information (S613).

[0160] In one embodiment, the processor (560) may obtain the user's health management information from a customer health management database. The user's health management information may include one or more of the user's identification information, the user's age, disease management items, and medication information.

[0161] The customer health management database may be included in the AI ​​server (500) or may be provided separately from the AI ​​server (500).

[0162] The processor (560) can generate a prompt to obtain a response result of a generation AI model provided in the generation AI server based on the user's health management information and questionnaire response information.

[0163] In one embodiment, the processor (560) may extract keywords from each of the user's health management information and questionnaire response information, and combine the extracted keywords to generate a single sentence. The processor (560) may obtain the generated single sentence as a prompt.

[0164] In one embodiment, the processor (560) may extract keywords using either text rank or topic modeling techniques.

[0165] Text Rank is a method of extracting keywords using a graph-based algorithm. It can be a method of representing sentences or words as graph nodes and analyzing the relationship between nodes to identify important words.

[0166] Topic modeling techniques can be a method of identifying major topics in text data and extracting keywords related to the identified topics.

[0167] The processor (560) can generate a prompt requesting a disease diagnosis based on health management information and questionnaire response information.

[0168] Prompts may include a combination of keywords and commands requesting information related to the disease.

[0169] For example, a prompt might be, "I have a history of high blood pressure, my blood sugar level is 135, and I need to investigate related medical conditions."

[0170] High blood pressure history may be a user's health management information, and blood sugar levels may be information from a questionnaire response.

[0171] In another embodiment, the processor (560) may generate a prompt based on the user's health management information, questionnaire response information, and medical information stored in a medical database. The medical database may be included in the AI ​​server (500) or may be provided separately from the AI ​​server (500).

[0172] Medical databases may contain medical information. This medical information may include one or more of the following: medical literature, medical terminology, clinical trial information, and medical statistics. Medical information may also include specialized medical knowledge.

[0173] The processor (560) can extract related medical terms from a medical DB based on one or more of the user's health management information or questionnaire response information.

[0174] For example, a prompt might be, "I have a history of high blood pressure, my blood sugar level is 135, and I need to investigate my insulin-related medical conditions."

[0175] Hypertension history may be a user's health management information, blood sugar level may be questionnaire response information, and insulin may be a medical term extracted from a medical database related to blood sugar level.

[0176] The processor (560) can generate prompts based on the user's health management information, questionnaire response information, and medical information. If medical terminology is used in the prompts, the user's illness can be more accurately predicted.

[0177] In one embodiment, the processor (560) may generate a prompt each time the user utters a voice command, or may utilize a pre-generated, fixed-format prompt.

[0178] The processor (560) may determine whether to use a fixed format prompt based on the analysis results of the user's voice command.

[0179] The processor (560) can store a plurality of previously generated prompts in the memory (530).

[0180] For example, pre-generated prompts may have fixed formats such as:

[0181] That is, the prompt may have a fixed format such as <user's past disease history + measured biomarker value + search for related diseases> or <user's current disease + measured biomarker value + search for related diseases>.

[0182] If the analysis result of the user's voice command is a diagnosis of a disease, the processor (560) can extract a previously generated prompt from the memory (530) and transmit the extracted prompt to the generation AI server (610).

[0183] In this way, by using a prompt generated based on the analysis results of a voice command, the amount of computation and time required to generate a prompt can be reduced.

[0184] The processor (560) of the AI ​​server (500) can transmit a prompt to the generation AI server (610) through the communication unit (510) (S615) and receive a generation response result from the generation AI server (610) (S617).

[0185] The generation AI server (610) may include one or more generation AI models.

[0186] Generative AI models can be artificial intelligence-based models that output responses to prompts, using deep learning or machine learning.

[0187] A generative AI model could be a large language model (LLM) trained to understand human language and generate responses based on input.

[0188] A large language model can be called a natural language generation AI model.

[0189] Natural language generation AI models can be trained using deep learning architectures such as recurrent neural networks (RNNs) and transformation models using attention mechanisms through large-scale text data.

[0190] Data for training natural language generation AI models can be collected from the web or various text sources.

[0191] A natural language generation AI model can take a received prompt as input and output generated response information.

[0192] The generated response results may include diagnostic results about the user's health.

[0193] The generated response result may include one or more of the following: the name of the disease, temporary measures, treatment subjects, and hospital information for treating the disease.

[0194] The processor (560) of the AI ​​server (500) can transmit the generation response result to the display device (100) through the communication unit (510) (S619).

[0195] The controller (170) of the display device (100) can output the generation response result (S621).

[0196] The controller (170) can display the generation response result on the display (180).

[0197] The controller (170) of the display device (100) can provide a hospital reservation service based on the generated response result (S623).

[0198] The controller (170) can determine the treatment subject included in the generated response result and reserve a hospital equipped with the treatment subject.

[0199] The controller (170) can reserve a hospital with medical services based on the user's location.

[0200] The controller (170) can search for a hospital with a medical department within a certain distance from the user's location and make a reservation at the searched hospital.

[0201] The controller (170) can display hospital reservation information on a display (180) according to the hospital reservation, or transmit the hospital reservation information to the user's mobile terminal through a wireless communication interface (173).

[0202] Hospital appointment information may include the hospital's appointment date and appointment time.

[0203] The controller (170) can output a notification about hospital reservation information before the hospital reservation date and reservation time expire.

[0204] In this way, according to embodiments of the present disclosure, accurate diagnostic results for the user's symptoms can be provided to the user as prompts are generated using questionnaire response information, such as additional utterances, responses to a questionnaire, or the results of the user performing a specific action.

[0205] In addition, according to an embodiment of the present disclosure, a reservation service is provided that automatically reserves a hospital for treatment of the user's disease, thereby greatly improving the convenience of the user's health management.

[0206] FIG. 8 is a drawing illustrating a process of identifying a user's symptoms and providing a remote diagnosis system according to an embodiment of the present disclosure.

[0207] Referring to FIG. 8, the customer health management DB (810) may store user health management information for each of a plurality of users.

[0208] The display device (100) can collect the user's health management information and transmit the collected user's health management information to the AI ​​server (500). The AI ​​server (500) can match the user's health management information to the user identification information and store it in the customer health management DB (810).

[0209] The process of collecting users' health management information is described below.

[0210] The health management DB (830) may store health management methods for responding to diseases, major disease types by age, and diet / exercise / cautions / medication information according to major diseases.

[0211] The health management DB (830) may further store a questionnaire. The questionnaire may be generated by the generation AI server (600).

[0212] The user The display device (100) can acquire the voice command and transmit the acquired voice command and user identification information to the AI ​​server (500).

[0213] The AI ​​server (500) can analyze the intent of a voice command and obtain analysis results.

[0214] If additional questionnaires are required based on the acquired analysis results, the AI ​​server (500) can transmit an additional questionnaire request to the display device (100).

[0215] Additional questionnaire requests may include one or more of a questionnaire, a request for additional utterances about symptoms, or a request for the results of performing a specific action.

[0216] The display device (100) can output an additional questionnaire request on the display (180).

[0217] The display device (100) can receive questionnaire response information in response to an additional questionnaire request and transmit the received questionnaire response information to the AI ​​server (500).

[0218] The AI ​​server (500) can store questionnaire response information in the customer health management DB (810) to update the user's health management information.

[0219] If an additional questionnaire request is required based on the questionnaire response information, the AI ​​server (500) may transmit an additional questionnaire request to the display device (100) again.

[0220] The AI ​​server (500) can generate a prompt requesting a health diagnosis of the user based on the user's health management information and questionnaire response information.

[0221] The AI ​​server (500) can transmit the generated prompt to the generation AI server (600). The generation AI server (600) can obtain a generated response result including a diagnosis result on the user's health for the prompt using a natural language generation AI model.

[0222] Diagnostic results may include the name of the disease, interim measures, preventive measures, and hospital appointment information.

[0223] The AI ​​server (500) can transmit the generated response result to the display device (100).

[0224] The display device (100) can display a notification window inquiring about the result of the generation response and whether to use the remote medical treatment service.

[0225] When the AI ​​server (500) receives a command from the display device (100) agreeing to use the remote medical treatment service, it can connect the remote medical treatment system (850) and the display device (100).

[0226] Users can receive remote medical services through a camera or microphone.

[0227] The results of remote medical treatment can be stored in the customer health management DB (810) and used to update the user's health management information.

[0228] FIG. 9 is a diagram illustrating a process of collecting user health management information according to one embodiment of the present disclosure.

[0229] Referring to FIG. 9, the display device (100) may display an account creation window (910) for creating a user account. The account creation window (910) may include input windows for collecting the user's ID, password, and date of birth.

[0230] The display device (100) can display a pop-up window (930) inquiring about settings for health care after user information is entered through the account creation window (910).

[0231] When the display device (100) receives an input for setting health care through a pop-up window (930), it can display a first health care setting menu (950).

[0232] The first health management setting menu (950) may be a menu for identifying disease items that the user has.

[0233] The display device (100) can display a second health management setting menu (970) when any one of the multiple disease items included in the first health management setting menu (950) is selected.

[0234] The second health management settings menu (970) may include health management items for the selected disease item. The health management items may include diet management items, exercise management items, and medication items.

[0235] The display device (100) can transmit the user's health management information and user identification information obtained through the first health management setting menu (950) and the second health management setting menu (970) to the AI ​​server (500).

[0236] User identification information may include the user's ID.

[0237] The AI ​​server (500) can match the received user's health management information to user identification information and store it in the customer health management DB (810).

[0238] FIG. 10 is a diagram showing an example of outputting a generation response result according to one embodiment of the present disclosure.

[0239] Referring to FIG. 10, the display device (100) can display a generated response result (1000) indicating the user's health diagnosis result from the AI ​​server (500) on the display (180).

[0240] The generated response result (1000) may include the name of the disease, symptoms of the disease, and measures for the disease.

[0241] The generated response result (1000) is a result based on the user's health management information and questionnaire response information, and may include a more accurate examination result than a diagnosis predicted using only biometric data.

[0242] FIG. 11A and FIG. 11B are diagrams illustrating an example of providing a hospital reservation service for treatment of a disease included in a generated response result according to one embodiment of the present disclosure.

[0243] Referring to FIG. 11a, the display device (100) can display a hospital reservation inquiry window (1110) on the display (180) that inquires about a reservation at a hospital that treats a disease included in the generated response result (1000) of FIG. 10.

[0244] When a consent input is received through the hospital reservation inquiry window (1110), the display device (100) can make a reservation at a hospital that treats the disease based on the user's location. The user's location may be the same as the location of the display device (100).

[0245] The display device (100) can obtain the location of the display device (100) as the user's location and search for a hospital that treats a disease of the generated response result (1000) within a certain distance from the user's location.

[0246] The display device (100) can obtain information on the treatment time of the searched hospital and make a reservation based on the obtained information on the treatment time of the hospital.

[0247] Accordingly, the display device (100) can display hospital reservation information (1130) on the display (180).

[0248] Hospital appointment information (1130) may include the name of the hospital, the location of the hospital, and the appointment time of the hospital.

[0249] The display device (100) can output a notification including hospital reservation information (1130) on the display (180) a certain amount of time before the hospital reservation date or the reservation time on the reservation date.

[0250] The display device (100) can transmit a notification including hospital reservation information (1130) to a mobile terminal such as a user's smartphone via a wireless communication interface (173) a certain amount of time before the hospital reservation time.

[0251] In another embodiment, instead of the display device (100), the AI ​​server (500) can generate hospital appointment information based on the generated response result.

[0252] The AI ​​server (500) can transmit the generated hospital reservation information to the display device (100), and the display device (100) can display the received hospital reservation information.

[0253] The AI ​​server (500) can receive the user's location from the display device (100) and search for a hospital that can treat a disease included in the diagnosis result adjacent to the received user's location.

[0254] The AI ​​server (500) can automatically perform a treatment reservation for treatment at the hospital by searching for the treatment hours and reservation times of the searched hospital.

[0255] When the reserved treatment time arrives, the AI ​​server (500) can transmit a control command to the display device (100) to cause the display device (100) to output a notification of the hospital reservation time.

[0256] In this way, according to an embodiment of the present disclosure, a hospital reservation service for treating a disease diagnosed to a user can be automatically provided.

[0257] Users can receive diagnosis and hospital reservation services for diseases by simply speaking or responding to a questionnaire or performing specific movements at home, enabling them to respond quickly to diseases and conveniently make hospital reservations.

[0258] According to one embodiment of the present invention, the above-described method can be implemented as processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0259] The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

Claims

1. In artificial intelligence devices, Memory that stores the user's health management information; A communication unit that communicates with an electronic device or a generation artificial intelligence (AI) server; and A processor comprising: a processor configured to receive a voice command uttered by a user from the electronic device, obtain an analysis result of the received voice command, and, if the obtained analysis result is a request for a health diagnosis of the user, transmit an additional questionnaire request to the electronic device through the communication unit, receive questionnaire response information in response to the additional questionnaire request from the electronic device through the communication unit, generate a prompt requesting a disease diagnosis based on the health management information and the questionnaire response information, transmit the generated prompt to the generation AI server through the communication unit, and receive a diagnosis result from the generation AI server through the communication unit. Artificial intelligence device.

2. In paragraph 1, The above processor Obtaining medical terms from a medical database and generating a prompt based on the health care information, the questionnaire response information, and the medical terms. Artificial intelligence device.

3. In paragraph 2, The above additional questionnaire request is A questionnaire, including one or more of the user's additional utterance requests or results requests for performing a specific action. Artificial intelligence device.

4. In paragraph 1, The above diagnostic results are Contains one or more of the following information: the name of the disease, the treatment, the department of treatment, or the hospital treating the disease. Artificial intelligence device.

5. In paragraph 4, The above processor Based on the above diagnosis results, the hospital is searched and a medical appointment is made through the searched hospital. Artificial intelligence device.

6. In paragraph 5, The above processor When the reservation date or reservation time included in the reservation information arrives, a control command is transmitted to the electronic device to cause the electronic device to output the reservation information. Artificial intelligence device.

7. In paragraph 1, The above processor Extract keywords from each of the above health management information and the above questionnaire response information, and generate the prompt based on the extracted keywords. Artificial intelligence device.

8. In the method of operating an artificial intelligence device, A step of receiving a voice command spoken by a user from an electronic device; A step of obtaining analysis results of a received voice command; If the obtained analysis result is a request for a health diagnosis of the user, a step of transmitting an additional questionnaire request to the electronic device; A step of receiving questionnaire response information in response to the above additional questionnaire request from the electronic device; A step of generating a prompt requesting a disease diagnosis based on health management information and the questionnaire response information; A step of transmitting the generated prompt to the generated AI server; and A step of receiving a diagnosis result from the above-mentioned generation AI server is included. How an artificial intelligence device operates.

9. In paragraph 1, The steps to create the above prompt are A method comprising: obtaining medical terms from a medical database; and generating a prompt based on the health care information, the questionnaire response information, and the medical terms. How an artificial intelligence device operates.

10. In paragraph 9, The above additional questionnaire request is A questionnaire, including one or more of the user's additional utterance requests or results requests for performing a specific action. How an artificial intelligence device operates.

11. In paragraph 8, The above diagnostic results are Contains one or more of the following information: the name of the disease, the treatment, the department of treatment, or the hospital treating the disease. How an artificial intelligence device operates.

12. In paragraph 11, A step of searching for the hospital based on the above diagnosis results; and Further comprising a step of obtaining reservation information of the searched hospital. How an artificial intelligence device operates.

13. In paragraph 12, Further comprising a step of transmitting a control command to the electronic device to cause the electronic device to output the reservation information when the reservation date or reservation time included in the reservation information arrives. How an artificial intelligence device operates.

14. In paragraph 8, The steps to create the above prompt are A step of extracting keywords from each of the health management information and the questionnaire response information, and generating the prompt based on the extracted keywords. How an artificial intelligence device operates.

Images