System

Abstract

A system is provided.SOLUTION: A system includes means for collecting voice data and search history data from a smart device of a user, means for converting the voice data into text data, means for analyzing the text data and the search history data and extracting a keyword indicating health abnormality, means for specifying an appropriate medical institution based on the keyword and acquiring an appointment available date and time, means for transmitting a notification including an appointment proposal to the smart device of the user, means for confirming an appointment in cooperation with an appointment system of the medical institution when the user accepts the appointment proposal, and means for notifying the smart device of the user of detailed information of the appointment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, it is important for individuals to visit a medical institution at the appropriate time when they experience health abnormalities or signs of illness. However, many people often postpone visiting a medical institution even when they are aware of a health abnormality, which results in the risk of their condition worsening. To solve this problem, there is a need for a system that can quickly detect health abnormalities that people are aware of or potentially feeling and automatically schedule appointments at appropriate medical institutions. [Means for solving the problem]

[0005] The present invention is a system that collects voice data and search history data from a user's smart device, analyzes the data, and extracts keywords indicating health abnormalities. Specifically, the system first converts the voice data into text data using a voice recognition function. Next, it extracts keywords related to health abnormalities from the analyzed text data and search history data, identifies an appropriate medical institution based on the keywords, and obtains an available appointment date and time. The system then notifies the user of the appointment proposal, and if the user accepts the proposal, it cooperates with the medical institution's appointment system to confirm the appointment. Finally, it notifies the user of the appointment details. The system also includes a function to send reminder notifications when the appointment date approaches. This series of processes provides a system that helps users promptly visit an appropriate medical institution when they sense a health abnormality.

[0006] A "smart device" is a portable or stationary electronic device that has internet connectivity, advanced computing capabilities, and can be operated by a user through voice or text input.

[0007] "Voice data" means a recording of a user speaking into a smart device, typically stored in the form of a digital file.

[0008] "Search history data" refers to data that indicates the history of search queries a user has made on the Internet, and is typically stored in text format.

[0009] "Speech recognition function" refers to the technology or algorithm that converts voice data into text data.

[0010] "Text data" refers to character information obtained as a result of converting voice data using a voice recognition function.

[0011] "Analysis" refers to the process of analyzing information contained in the text data and search history data and extracting keywords that indicate health abnormalities.

[0012] "Health abnormality" refers to a condition or symptom that indicates an abnormality in the user's physical condition.

[0013] "Keywords" are important words or phrases related to health abnormalities extracted by analyzing the voice data and search history data.

[0014] "Medical institutions" are facilities that provide medical services, such as hospitals, clinics, and dental clinics.

[0015] The "available appointment dates and times" are the dates and times when the medical institution is available to accept the user for treatment.

[0016] An "appointment suggestion" is a guide presented to the user by the system for making an appointment at a medical institution.

[0017] A "notification" is a message sent to a smart device by a system to convey information to a user.

[0018] A "reservation system" is a system for managing reservations at medical institutions and for confirming and adjusting reservations.

[0019] "Confirming a reservation" is the process in which the date and time of the consultation at the medical institution is officially decided after the user accepts the proposal.

[0020] "Detailed information" is specific information about the appointment, such as the date and time of the appointment, the name and location of the medical institution.

[0021] A "reminder notification" is a notification sent to a user when an appointment is approaching, to help the user reconfirm the appointment date and time. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2]1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] As an embodiment of the present invention, the system program is designed as follows.

[0044] Overall overview

[0045] This automated system collects voice data and search history data from users' smart devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of terminals, servers, and users.

[0046] System Configuration and Operation

[0047] Data collection

[0048] The device collects voice input and search history every time a user speaks to or searches on the smart device. For example, if a user says "my tooth hurts" to their smartphone, the device records the voice data. At the same time, if the user searches for keywords such as "toothache causes" on the browser, the device also collects that history data.

[0049] Data Conversion and Transmission

[0050] The collected voice data is converted into text data using the device's voice recognition function, and this converted text data and search history data are then sent to the server.

[0051] Data analysis

[0052] The server analyzes the submitted text data and search history data. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "toothache" or "tooth decay" are recognized, the server will determine that this is a type of health abnormality.

[0053] Identifying medical institutions and suggesting appointments

[0054] Based on the analysis results, the server takes into account the user's current location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[0055] User Notification

[0056] The device displays the appointment suggestion notification sent from the server to the user. For example, a message such as "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?" can be displayed. The user can choose to respond with "Yes" or "No" to this message.

[0057] Confirmation of reservation and notification of details

[0058] If the user answers "yes," the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) are notified to the terminal.

[0059] Reminder function

[0060] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[0061] Specific examples

[0062] 1. The user says to their smartphone, "I've been having stomach aches lately."

[0063] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[0064] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[0065] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[0066] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[0067] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[0068] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[0069] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[0070] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0071] Through this series of processes, users can quickly recognize any health abnormalities and ensure that they are referred to an appropriate medical institution.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] The user speaks a health-related phrase (e.g., "My tooth hurts") into their smart device.

[0075] Step 2:

[0076] The device uses voice recognition to capture the user's voice data and convert it into text data.

[0077] Step 3:

[0078] The device retrieves the user's search history data from the browser.

[0079] Step 4:

[0080] The terminal transmits the converted text data and the acquired search history data to the server.

[0081] Step 5:

[0082] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[0083] Step 6:

[0084] The server searches for an appropriate medical institution using the user's location information based on the extracted keywords.

[0085] Step 7:

[0086] The server obtains the available appointment dates and times for the medical institutions found by the search.

[0087] Step 8:

[0088] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[0089] Step 9:

[0090] The device displays a notification message from the server to the user (e.g., "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?").

[0091] Step 10:

[0092] The user responds to the notification with a "yes" or "no" answer.

[0093] Step 11:

[0094] The terminal sends the user's response results to the server.

[0095] Step 12:

[0096] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[0097] Step 13:

[0098] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[0099] Step 14:

[0100] The terminal displays a reservation confirmation message from the server to the user.

[0101] Step 15:

[0102] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[0103] Step 16:

[0104] The device displays a reminder notification from the server to the user.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] In modern society, there are an increasing number of situations where users need to quickly understand their own health status and seek medical attention at an appropriate medical institution. However, users often need a great deal of time and effort to collect medical information, and there is a lack of appropriate systems for selecting and booking medical institutions. In particular, the time required for early detection of health abnormalities and selecting an appropriate medical institution is an issue.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for collecting voice data and search history data from the user's electronic device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicative of health abnormalities, means for identifying an appropriate medical institution based on the keywords and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's electronic device, means for cooperating with the medical institution's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's electronic device of detailed information about the appointment, and means for generating and sending a reminder notification when the appointment date and time approaches. This allows the user to quickly understand their own health abnormalities and ensure that they visit an appropriate medical institution.

[0110] "User" refers to an individual or organization that uses an electronic device.

[0111] "Electronic terminal" refers to devices with communication capabilities, such as smart devices, personal computers, and tablets.

[0112] "Audio data" refers to digital data acquired as audio input.

[0113] "Search History Data" refers to data that includes the history of searches you perform on your internet browser or applications.

[0114] "Text data" refers to data in which voice data has been converted into a string of characters.

[0115] "Health abnormalities" refers to any abnormal or unwell health condition experienced by the User.

[0116] "Keywords" refer to specific words or phrases related to a health condition.

[0117] "Medical institution" refers to a facility that provides medical services, such as a doctor's office, hospital, or clinic.

[0118] "Available appointment dates" refers to dates and times when a medical institution can accept appointments for medical treatment.

[0119] "Notification" means the sending of a message or alert to a User's electronic device.

[0120] "Reminder notification" refers to notifying the user again when the reservation deadline approaches.

[0121] "Server" refers to a computer system that stores, manages, analyzes, and communicates data over a network.

[0122] As an embodiment of the present invention, the system program is implemented as follows.

[0123] System Overview

[0124] This automated system collects voice data and search history data from users' electronic devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of three elements: the device, the server, and the user.

[0125] Data collection and transformation

[0126] The device collects voice input and search history every time a user speaks to the device or searches. For example, if a user says to the device, "I've had a stomach ache recently," the device records the voice data. Similarly, if a user searches for "causes of stomach ache" on a browser, the device also collects that history data.

[0127] The collected voice data is converted into text data using the device's voice recognition function (e.g., a cloud-based voice recognition API). Specifically, the Google Cloud Speech-to-Text API can be used. This converted text data and search history data are then sent to a server.

[0128] Data analysis

[0129] The server analyzes the text data and search history data sent to it. Natural language processing (NLP) technology is used for the analysis. For example, keywords indicating health abnormalities are extracted using the Google Cloud Natural Language API. Specifically, keywords such as "my stomach hurts" and "causes of stomach pain" are extracted from the text data and search history data, and the presence or absence of health abnormalities is determined based on this.

[0130] Identifying medical institutions and suggesting appointments

[0131] Based on the analysis results, the server takes into account the user's current location information (e.g., GPS data) and searches for suitable nearby medical institutions. It then obtains available appointment dates and times for each medical institution and creates an appointment proposal. The appointment proposal includes a specific message such as "Would you like to make an appointment at a nearby internal medicine clinic?" The proposal is then sent to the device as a notification.

[0132] User notification and booking confirmation

[0133] The terminal displays the appointment suggestion notification sent from the server to the user. If the user responds "Yes" to this message, the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the terminal is notified of the details (date, time, location, name of the medical institution, etc.).

[0134] Reminder function

[0135] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[0136] Specific examples

[0137] 1. The user speaks to an electronic device saying, "I've been having stomach aches lately."

[0138] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[0139] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[0140] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[0141] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[0142] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[0143] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[0144] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[0145] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0146] Prompt Sentence Examples

[0147] Example prompt sentence:

[0148] Please describe a system that detects health abnormalities and schedules appropriate medical appointments by having users talk to their smartphones or look at their search history. Please include the following information: the hardware and software used, what data is collected and analyzed, how it is collected and analyzed, how users are notified, and specific examples.

[0149] Through the above-described procedures, the present invention assists the user in managing his or her health and ensures prompt and appropriate medical treatment.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1: Data collection

[0152] A user speaks to an electronic device or performs a search. The user's voice data and search history data are obtained as input. The device records the voice data when the user speaks to the electronic device, such as "I've had a stomachache recently," and collects search history data for "causes of stomach pain" on the browser. The recorded voice data and search history data are generated as output.

[0153] Step 2: Data conversion

[0154] The device converts the collected voice data into text data using a speech recognition function (for example, Google Cloud Speech-to-Text API). The recorded voice data is given as input. As a result of the conversion, the text data "I've had a stomach ache recently" is output. Furthermore, the search history data is used in its original form.

[0155] Step 3: Sending data

[0156] The terminal transmits the converted text data and search history data to the server. As input, the text data and search history data are provided. These data are encrypted and transmitted to the server. As output, the text data and search history data received by the server are obtained.

[0157] Step 4: Data analysis

[0158] The server analyzes the received text data and search history data to extract keywords that indicate health abnormalities. As input, the received text data "I've had a stomach ache recently" and search history data "Causes of stomach pain" are given. The server uses the Google Cloud Natural Language API to extract keywords such as "my stomach hurts" and "stomach pain" from these data. The extracted keywords are generated as output.

[0159] Step 5: Identify medical facilities

[0160] The server determines health abnormalities based on the extracted keywords and identifies appropriate medical institutions. The extracted keywords and the user's current location information (e.g., GPS data) are given as input. The server searches for nearby medical institutions and obtains available appointment dates and times. The output is the medical institution and available appointment date and time information.

[0161] Step 6: Creating a booking proposal

[0162] The server creates an appointment suggestion based on the retrieved medical institution and available appointment date and time. Medical institution information and available appointment date and time information are given as input. The server creates an appointment suggestion such as "Would you like to make an appointment at a nearby internal medicine clinic?". As output, an appointment suggestion notification is generated.

[0163] Step 7: Send booking proposal notification

[0164] The server sends the created reservation proposal notification to the terminal. The reservation proposal notification is given as input. The server sends the reservation proposal notification to the user's terminal. The reservation proposal notification received by the terminal is obtained as output.

[0165] Step 8: Receiving user response

[0166] The user responds to the reservation suggestion notification displayed on the terminal. As input, the user's response "yes" or "no" is obtained. For example, the user responds "yes." As output, the user's response is generated.

[0167] Step 9: Confirm your booking

[0168] The server receives the user's response and confirms the appointment by linking with the medical institution's appointment system. The user's response and the medical institution's appointment information are given as input. The server confirms the appointment for the specified date and time through the medical institution's online appointment system. The detailed appointment information is generated as output.

[0169] Step 10: Send reservation details notification

[0170] The server sends the details of the confirmed reservation to the terminal. As input, the reservation details are given. The server sends the reservation details to the user's terminal. As output, a reservation details notification received by the terminal is obtained.

[0171] Step 11: Generate a reminder notification

[0172] The server generates a reminder notification when the reservation date and time is approaching. Reservation date and time information is given as input. The server generates the reminder notification. The generated reminder notification is obtained as output.

[0173] Step 12: Send reminders

[0174] The server sends the generated remind notification to the terminal. The remind notification is given as input. The server sends the remind notification to the user's terminal. The output is the remind notification received by the terminal.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Conventional systems lack the ability to quickly analyze a user's health status and recommend appropriate medical institutions and health-related products. As a result, users have no means to respond immediately to their health status and spend a lot of time and effort finding appropriate medical services and health products. The present invention aims to solve this problem by providing a system that allows users to quickly recognize health abnormalities and receive recommendations for appropriate medical institutions and health-related products.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes means for collecting voice data and search history data from the user's computer terminal, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting words or phrases indicating health abnormalities, means for identifying an appropriate medical facility based on the words or phrases and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's computer terminal, means for coordinating with the medical facility's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's computer terminal of details of the appointment, and means for analyzing the user's health condition data and suggesting health-related products. This allows the user to quickly respond to their health condition and receive appointments at appropriate medical institutions and suggestions for health-related products.

[0180] A "user" is a person who uses this system and provides voice data and search history data.

[0181] "Computer terminal" means a device used by a user, and is a digital device including a smartphone, tablet, PC, smart glasses, etc.

[0182] "Voice data" refers to voice information uttered by a user into a computer terminal and analyzed using voice recognition technology.

[0183] "Search history data" refers to the history of searches a user has conducted on the Internet, and is used to infer the user's interests and health status.

[0184] "Character data" is information obtained by analyzing voice data and converting it into text.

[0185] "Terms" are keywords and phrases extracted from voice data and search history data, and are used to identify health abnormalities and suggest products.

[0186] A "medical facility" is a facility where a person can receive medical treatment from a doctor, and refers to general forms of medical institutions, including hospitals and clinics.

[0187] The "available appointment dates and times" are specific dates and times when a medical facility can accept a user's appointment.

[0188] "Reservation proposal" refers to the content of a proposal for a medical facility reservation that the server sends to the user based on the analysis results.

[0189] "Health Status Data" refers to information related to a user's health that is inferred based on goad cues obtained from voice data and search history data.

[0190] "Health-related products" are products suggested based on the user's health condition, including supplements, medicines, health foods, etc.

[0191] Overall overview

[0192] As an embodiment of the present invention, the system program is designed as follows.

[0193] System Configuration and Operation

[0194] Data collection

[0195] First, the user inputs their health condition into a computer terminal by voice. For example, the user might say, "I've been feeling tired lately." This voice data is converted into text data by voice recognition software (e.g., Google Cloud Speech-to-Text API) running on the computer terminal. At the same time, if the user searches online for something like "ways to recover from fatigue," that search history data is also collected.

[0196] Data analysis

[0197] The server receives the converted text data and search history data and analyzes it using natural language processing (NLP) technology (e.g., Transformers with the BERT model). This analysis extracts words that indicate health abnormalities. For example, the analysis results may include words such as "fatigue" and "rest."

[0198] Identifying and recommending medical facilities and products

[0199] Based on the analysis results, the server takes into account the user's location information and identifies appropriate medical facilities and health-related products, such as nearby internal medicine clinics or supplements effective for fatigue recovery. This information is then sent to the computer terminal along with available appointment times and detailed product information.

[0200] User Notification and Response

[0201] The user receives information about the proposed medical facility and product on the computer terminal. For example, a message such as "Would you like to purchase a fatigue recovery supplement?" or "I have made an appointment with a nearby internal medicine clinic. Would you like to confirm the appointment?" is displayed. If the user accepts the appointment suggestion, the server connects with the medical facility's reservation system to confirm the appointment. If the user responds to the product suggestion, the purchase procedure is carried out.

[0202] Reservation details and reminders

[0203] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. For example, it may say, "Your reservation has been made at the internal medicine clinic on a certain date at a certain time." In addition, when the reservation date and time approaches, a reminder notification is generated and sent again to the computer terminal to notify the user.

[0204] Specific examples

[0205] A user says to their smartphone, "I have a headache."

[0206] 1. Speech recognition software converts the speech into the text "I have a headache."

[0207] 2. The converted character data is sent to the server.

[0208] 3. The server analyzes the text data and search history data to extract words that indicate health abnormalities, such as "headache."

[0209] 4. Based on the analysis results, medical facilities and health products related to headaches are identified, and detailed information is sent from the server to the smartphone.

[0210] 5. The user responds to prompts such as "Would you like to make an appointment with a local doctor?" or "Would you like to buy some headache medicine?"

[0211] 6. Once the user approves the reservation, the server connects to the medical facility's reservation system to confirm the reservation.

[0212] 7. The server will notify your smartphone of the reservation details.

[0213] 8. When your appointment time approaches, a reminder notification will be sent to your smartphone.

[0214] Prompt Sentence Examples

[0215] "Please generate an application program that analyzes the user's health-related voice data and suggests appropriate medical institutions and health products. For example, if a user says, 'I have a headache,' convert the voice into text, analyze that data, and suggest related products."

[0216] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0217] Step 1:

[0218] A user inputs voice into a computer terminal. At this time, the computer terminal uses voice recognition software (e.g., Google Cloud Speech-to-Text API) to collect the voice data and convert it into text data. The input is the user's voice data, and the output is the converted text data.

[0219] Step 2:

[0220] The terminal sends the converted text data to the server. At the same time, the user's recent search history data is also sent to the server. The input is the text data and the search history data, and the output is the data sent to the server.

[0221] Step 3:

[0222] The server analyzes the received text data and search history data using natural language processing (NLP) techniques (e.g., Transformers with the BERT model). This analysis extracts phrases that indicate health abnormalities. The input is the text data and search history data, and the output is the extracted health abnormality phrases.

[0223] Step 4:

[0224] The server identifies appropriate medical facilities based on the analysis results and takes into account the user's location information. At the same time, it searches the database for health-related products related to the health abnormality. The input is the health abnormality phrase and location information, and the output is a list of identified medical facilities and health-related products.

[0225] Step 5:

[0226] The server creates reservation proposals and product proposals, including available appointment dates and times and detailed product information, and notifies the user of these proposals. The input is detailed information about medical facilities and products, and the output is a notification to the user.

[0227] Step 6:

[0228] The user checks the proposed content on the computer terminal and responds, for example, by answering "yes" or "no" to messages such as "Would you like to make a reservation?" or "Would you like to purchase a product?" The input at this step is the user response, and the output is the user's choice.

[0229] Step 7:

[0230] If the user accepts the reservation proposal, the server will confirm the reservation in cooperation with the medical facility's reservation system. If the user accepts the product proposal, the server will proceed with the purchase procedure. The input is the user's selection, and the output is the confirmed reservation and a notification of purchase completion.

[0231] Step 8:

[0232] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. Furthermore, when the reservation date and time approaches, a reminder notification is generated and sent to the user's computer terminal. The input is the confirmed reservation information, and the output is the reservation details notification and the reminder notification.

[0233] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0234] In one embodiment of the present invention, we design an automated system that collects voice data and search history data from a user's smart device, analyzes this data to detect health abnormalities, recommends appropriate medical institutions, and schedules appointments. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to perform additional analysis and judgment based on the user's emotional state.

[0235] Overall overview

[0236] This system consists of a terminal, a server, and a user. The terminal is a smart device that collects voice data and search history data. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. An emotion engine has also been added to evaluate the user's emotional state and complement the health abnormality analysis results.

[0237] System Configuration and Operation

[0238] Data collection

[0239] The device collects voice data and search history data every time the user speaks to the smart device or searches the internet. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device will record the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device will also collect that search history data.

[0240] Data Conversion and Transmission

[0241] The collected voice data is converted into text data using the device's voice recognition function, and the converted text data and search history data are sent to the server.

[0242] Data analysis

[0243] The server analyzes the text data and search history data sent to it. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[0244] Emotion recognition and evaluation

[0245] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine evaluates the user's emotional state based on their vocabulary, tone of voice, search results, etc. For example, if a user says, "I'm having a really hard time sleeping," the engine detects their feelings of confusion and anxiety.

[0246] Identifying medical institutions and suggesting appointments

[0247] Based on the analysis results and emotion evaluation results, the server searches for suitable nearby medical institutions, taking into account the user's location information. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[0248] User Notification

[0249] The device displays a notification of the appointment suggestion sent from the server to the user. For example, a message such as "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?" can be displayed. The user can respond to this message with "Yes" or "No."

[0250] Confirmation of reservation and notification of details

[0251] If the user answers "yes," the server will connect to the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) will be notified to the terminal.

[0252] Reminder function

[0253] When the appointment time approaches, the server generates a reminder notification and sends it to the device, which displays the reminder notification to the user to remind them not to forget to come to the appointment.

[0254] Specific examples

[0255] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[0256] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I haven't been sleeping well lately," to the server.

[0257] 3. At the same time, the user's browser search history for "insomnia relief" is also sent to the server.

[0258] 4. The server analyzes keywords such as "can't sleep well" and "insomnia" and also evaluates the user's emotional state.

[0259] 5. Based on the analysis results, search for nearby sleep clinics and obtain available appointment dates and times.

[0260] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[0261] 7. If the user responds "yes," the server confirms the appointment with the designated sleep clinic.

[0262] 8. The terminal displays detailed information such as "An appointment has been made for the sleep clinic at XX / XX / XX at XX time."

[0263] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0264] The present invention, which combines emotion engines in this way, enables quick responses to the user's health condition, selection of an appropriate medical institution, and support for consultation.

[0265] The processing flow will be explained below.

[0266] Step 1:

[0267] The user speaks a health-related phrase into their smart device (e.g., "I haven't been sleeping well lately").

[0268] Step 2:

[0269] The device uses voice recognition to capture the user's voice data and convert it into text data.

[0270] Step 3:

[0271] The device retrieves the user's search history data from the browser.

[0272] Step 4:

[0273] The terminal transmits the converted text data and the acquired search history data to the server.

[0274] Step 5:

[0275] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[0276] Step 6:

[0277] The server uses an emotion engine to recognize the user's emotion from the voice data and text data and evaluates the user's emotional state.

[0278] Step 7:

[0279] The server evaluates the user's health abnormalities and determines the risk based on the extracted keywords and sentiment analysis results.

[0280] Step 8:

[0281] The server references the user's location information and searches for appropriate medical facilities nearby.

[0282] Step 9:

[0283] The server obtains the available appointment dates and times for the medical institutions found by the search.

[0284] Step 10:

[0285] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[0286] Step 11:

[0287] The device displays a notification message from the server to the user (e.g., "You haven't been sleeping well lately. Would you like to make an appointment with a nearby sleep clinic?").

[0288] Step 12:

[0289] The user responds to the notification with a "yes" or "no" answer.

[0290] Step 13:

[0291] The terminal sends the user's response results to the server.

[0292] Step 14:

[0293] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[0294] Step 15:

[0295] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[0296] Step 16:

[0297] The terminal displays a reservation confirmation message from the server to the user.

[0298] Step 17:

[0299] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[0300] Step 18:

[0301] The device displays a reminder notification from the server to the user.

[0302] Specifically, when a user says, "I haven't been sleeping well lately," the device converts the voice data into text and sends it to the server. The server analyzes the text data, extracts keywords such as "can't sleep" and "insomnia," and uses an emotion engine to evaluate the user's feelings of anxiety and confusion. Based on this, the device searches for appropriate medical institutions and their available appointment dates and times, and makes a reservation suggestion. If the user responds "yes," the reservation is confirmed. The reservation details and a reminder notification are then sent to the user.

[0303] Example 2

[0304] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0305] In modern society, it is extremely important for users to take prompt and appropriate measures regarding their health. However, the process of users recognizing their own health abnormalities and finding an appropriate medical institution is complicated and time-consuming. Furthermore, a user's emotional state and stress level also have a significant impact on their health, but there is a lack of responses that take these into consideration. Therefore, there is a need for a system that efficiently utilizes a user's voice data and search history data to detect health abnormalities, identify appropriate medical institutions, and assist in making appointments, while also taking their emotional state into consideration.

[0306] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting voice data and digital internet information from the user's information processing device; means for converting the voice data into text data; means for analyzing the text data and digital internet information and extracting an identifier indicating a health abnormality; means for identifying an appropriate medical facility and obtaining an available appointment date and time based on the identifier; means for sending a notification including an appointment proposal to the user's information processing device; means for confirming the appointment in cooperation with the medical facility's appointment system if the user accepts the appointment proposal; means for notifying the user's information processing device of details of the appointment; and means including an emotion engine that recognizes the user's emotional state and performs complementary analysis based on the analysis results. This allows the user to respond to their health condition quickly and appropriately, and accurate detection of health abnormalities and recommendations for medical institutions are realized taking the user's emotional state into consideration.

[0307] "User's information processing device" refers to an electronic device used by a user, such as a smartphone, tablet, or personal computer, that has the function of collecting and transmitting voice data and digital internet information.

[0308] "Voice data" refers to data that is recorded in digital format via a microphone of what a user says to an information processing device.

[0309] "Digital Internet information" refers to historical data such as the search history a user has conducted on the Internet and the web pages they have accessed.

[0310] The "means for converting into text data" refers to a speech recognition technology or system for converting voice data into text format.

[0311] "Identifiers" are keywords or phrases extracted from the data being analyzed that are used to indicate specific health abnormalities.

[0312] "Medical facilities" are medical institutions such as hospitals and clinics that provide medical treatment according to the user's health condition.

[0313] An "emotion engine" is a technology or system that recognizes a user's emotional state based on their voice data and text data, and provides analysis results according to their emotions.

[0314] A "reservation system" is a system used by medical facilities to manage reservations, and has functions for accepting, managing, and confirming reservations.

[0315] A "notification" is a message sent from the server to the user's information processing device, and includes information such as a medical facility reservation suggestion, detailed reservation information, and a reminder.

[0316] "Emotional state" refers to the psychological state analyzed from the user's voice and text data, and indicates emotions such as joy, sadness, anger, and anxiety.

[0317] "Complementary analytics" refers to additional or detailed analytics based on a user's emotional state that can be used to make more accurate decisions or recommendations.

[0318] As an embodiment of the present invention, we have designed an automated system that collects voice data and digital internet information from a user's information processing device, analyzes this data to detect health abnormalities, recommends appropriate medical facilities, and makes appointments. This system also incorporates an emotion engine that recognizes the user's emotional state, allowing it to perform additional analysis and judgment based on the user's emotional state.

[0319] Overall overview

[0320] This system consists of the user's information processing device (smart device), a server, and the user's components. The user's information processing device collects voice data and digital internet information. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. In addition, an emotion engine has been added to evaluate the user's emotional state and complement the analysis results of health abnormalities.

[0321] Hardware and software used

[0322] Information processing equipment: smart devices (smartphones, tablets, etc.)

[0323] Server: Dedicated server for data analysis

[0324] Speech recognition technology: Google Speech-to-Text API, etc.

[0325] NLP engines: spaCy and NLTK built in Python

[0326] Emotion recognition engine: Uses Praat and openSMILE as voice analysis tools

[0327] Geographic information service: Google Maps API

[0328] Push notification service: Firebase Cloud Messaging

[0329] Natural language description of the process

[0330] The device collects voice data when the user speaks to the smart device and the search history data they perform on the Internet. For example, if a user says, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device collects the search history.

[0331] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are then sent to a server. The server then uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[0332] The server uses an emotion engine to recognize the user's emotional state from text and voice data. For example, if someone says, "I'm having a really hard time sleeping," the server can detect feelings of confusion and anxiety. This emotion assessment is used to complement the analysis results.

[0333] Based on the analysis results and emotion evaluation results, the server takes into account the user's location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates an appointment suggestion. The device displays the appointment suggestion notification sent from the server to the user. For example, a message may appear saying, "You seem to be having trouble sleeping. Would you like to make an appointment at a nearby sleep clinic?" The user can respond to this message with "Yes" or "No."

[0334] If the user responds "Yes," the server will link with the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, detailed information (date, time, location, name of medical institution, etc.) will be notified to the device. When the appointment date and time approaches, the server will generate a reminder notification and send it to the device. The device will display this reminder notification to the user to remind them not to forget to visit.

[0335] Specific examples

[0336] Example 1:

[0337] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[0338] 2. The device converts the speech into text using a speech recognition function (for example, Google Speech-to-Text API) and sends the text data, such as "I haven't been sleeping well lately," to the server.

[0339] 3. At the same time, the user's browser search history data for "insomnia relief" is sent to the server.

[0340] 4. The server uses a Python NLP engine (e.g., spaCy) to analyze keywords such as "can't sleep well" and "insomnia" and also evaluate the user's emotional state.

[0341] 5. Based on the analysis results, use the Google Maps API to search for nearby sleep clinics and obtain available appointment dates and times.

[0342] 6. Using Firebase Cloud Messaging, display a notification on the device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[0343] 7. If the user answers "Yes," the server calls the medical institution's reservation API to confirm the appointment with the specified sleep clinic.

[0344] 8. The device will display detailed information such as "Your appointment has been made at the sleep clinic on XX / XX / XX at XX time." A reminder notification will also be scheduled.

[0345] Prompt Sentence Examples

[0346] Please explain the process of suggesting an appointment for a sleep clinic to a user who says, "I haven't been sleeping well lately."

[0347] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0348] Step 1:

[0349] The device collects voice data when the user speaks to the smart device and the search history data of the user's internet searches. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "insomnia relief," the device collects the search history.

[0350] Input: User voice data, internet search history data

[0351] Output: Recorded voice data, collected search history data

[0352] Step 2:

[0353] The collected voice data is converted into text data using the device's voice recognition function. For example, the Google Speech-to-Text API is used to convert the speech "I haven't been sleeping well lately" into text. The converted text data and search history data are then sent to the server.

[0354] Input: Recorded audio data

[0355] Output: Text data converted to character data, search history data

[0356] Step 3:

[0357] The server analyzes the text data and search history data sent to it. It uses natural language processing (NLP) technology, such as a Python NLP engine (spaCy or NLTK), to extract keywords that indicate health abnormalities. For example, it could extract keywords such as "not sleeping well" or "insomnia" and determine that the user has sleep problems.

[0358] Input: Text data converted to character data, search history data

[0359] Output: Extracted keywords related to health abnormalities

[0360] Step 4:

[0361] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine uses speech analysis tools (Praat and openSMILE) to analyze the intonation, speed, and volume of the voice data and extract emotions from the text data. For example, if a user says, "I'm having a really hard time sleeping," the emotion engine will detect confusion and anxiety.

[0362] Input: Text data converted to character data, audio data

[0363] Output: Data about the user's emotional state

[0364] Step 5:

[0365] Based on the analysis results and emotion evaluation results, the server searches for appropriate nearby medical institutions, taking into account the user's location information. For example, it uses the Google Maps API to search for nearby sleep clinics based on the user's location information and obtains available appointment dates and times at each medical institution.

[0366] Input: location information, data analysis results, emotion evaluation results

[0367] Output: List of suitable medical institutions, available appointment dates and times

[0368] Step 6:

[0369] The device displays the suggestions for available medical institutions sent from the server as a notification to the user. For example, it uses Firebase Cloud Messaging to notify the user with a message such as, "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?"

[0370] Input: List of suitable medical institutions, available appointment dates and times

[0371] Output: Notification of booking proposal to user

[0372] Step 7:

[0373] If the user accepts the reservation proposal, the server cooperates with the medical institution's reservation system to confirm the reservation at the specified date and time, for example, through the medical institution's reservation API.

[0374] Input: User response

[0375] Output: Confirmed reservation information

[0376] Step 8:

[0377] After the reservation is successfully made, the details (date, time, location, name of medical institution, etc.) are notified to the device. In addition, when the reservation date and time approaches, the server generates a reminder notification and sends it to the device. The device displays this reminder notification to the user.

[0378] Input: Confirmed reservation information, schedule information for reminder notifications

[0379] Output: Reservation details notification, reminder notification

[0380] (Application example 2)

[0381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0382] Conventional health management systems and food delivery services have been unable to effectively utilize users' voice data and search history data to detect health abnormalities or recommend appropriate medical institutions. Furthermore, they have not made meal suggestions that take into account the user's emotional state, making it difficult for users to select meals based on their own health condition and emotions. This has led to the issue of users being unable to receive appropriate support to maintain and improve their health.

[0383] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0384] In this invention, the server includes means for collecting voice data and search history data from the user's information processing device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicating health abnormalities, means for identifying appropriate medical institutions based on the keywords and obtaining available appointment dates and times, means for suggesting appropriate meals based on the user's health abnormalities and emotional state, and means for transmitting the contents of the suggested meals to the information processing device. This allows the user to receive meal suggestions based on their health and emotional state, and also enables them to quickly make appointments at necessary medical institutions.

[0385] An "information processing device" is an electronic device used by a user, such as a smartphone, tablet, or computer, and is a device for collecting and analyzing voice data and search history data.

[0386] "Voice data" refers to digital data that records what a user says to an information processing device.

[0387] "Search history data" refers to data that records the history of a user's internet searches.

[0388] "Text data" refers to data obtained by converting voice data into character information.

[0389] "Keywords" are important words that indicate health abnormalities or emotional states, extracted from text data and search history data.

[0390] A "medical institution" is an institution that provides medical services, such as a hospital or clinic.

[0391] An "emotion engine" is software that analyzes a user's emotional state from their voice and text data.

[0392] "Meal suggestions" are information that recommends optimal meals based on the user's health and emotional state.

[0393] A "notification" is a message or an alert sent to an information processing device.

[0394] A "reservation system" is a system for managing and processing reservations at medical institutions.

[0395] As an embodiment of the present invention, a system applied to a food delivery service consisting of a user, a terminal, and a server will be described. The operation of each element and the overall flow will be described in detail below.

[0396] Configuration and Operation

[0397] Data collection

[0398] The device collects voice data entered by the user using the voice recognition function. For example, if the user says, "I've been feeling tired lately," the device records the voice and saves it as voice data. At the same time, it collects the keywords the user searched for (e.g., "how to relieve fatigue") as search history data.

[0399] Data Conversion and Transmission

[0400] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are sent to the server. For voice recognition, the speech_recognition library, for example, is used.

[0401] Data analysis

[0402] The server uses natural language processing (NLP) technology to analyze the submitted text data and search history data, extracting keywords that indicate health abnormalities (e.g., "fatigue" and "easily tired"). The analysis uses a health API (e.g., https: / / myhealthapi.example.com).

[0403] Emotion recognition and evaluation

[0404] The server identifies the user's emotional state using an emotion engine. This emotion engine evaluates the user's emotion from text data and voice tone. For emotion analysis, an emotion analysis API (e.g., https: / / myfeelingsapi.example.com) is used.

[0405] Meal suggestions

[0406] Based on the analyzed health and emotion data, the server suggests appropriate meals using a meal suggestion API (e.g., https: / / mydeliveryapi.example.com).

[0407] User Notification

[0408] The device will notify the user of the meal suggestions sent from the server, for example, by displaying a message saying, "We will suggest nutritious meals that suit your condition."

[0409] Specific examples

[0410] For example, if a user says, "I've been feeling a bit tired lately," the system analyzes the voice data and determines that the user is tired. Next, it performs emotion analysis to evaluate the user's emotions. Based on this, it suggests nutritious meals (e.g., vitamin-rich meals) and notifies the information processing device so that the user can immediately order the meals.

[0411] Prompt Sentence Examples

[0412] "User voice data: 'I've been feeling a bit tired lately.' Search history: 'How to relieve fatigue.' Analyze this data to suggest nutritious meals that are suitable for the user."

[0413] In this way, users can receive optimal support based on their health and emotional state.

[0414] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0415] Step 1:

[0416] The device collects voice data entered by the user using the voice recognition function. The input requires the user's voice, such as saying, "I've been feeling tired lately." The voice data is stored as digital data on the device. In addition, if the user enters search keywords such as "how to relieve fatigue," the device also collects search history data.

[0417] Step 2:

[0418] The device converts the collected voice data into text data using a voice recognition function. Digital voice data is used as input, and the conversion is performed using, for example, the speech_recognition library. The output is string data that can be analyzed from the voice data. At the same time, the converted text data and search history data are sent to the server.

[0419] Step 3:

[0420] The server analyzes the submitted text data and search history data using natural language processing (NLP) techniques. The input is the text data and search history data, and keywords (e.g., "fatigue" and "easily tired") are extracted using a health API (e.g., https: / / myhealthapi.example.com). The extracted keywords are obtained as output.

[0421] Step 4:

[0422] The server uses an emotion engine to analyze the user's emotions from the text data. It uses the text data obtained in step 3 as input and utilizes an emotion analysis API (e.g., https: / / myfeelingsapi.example.com). It outputs the user's emotional state (e.g., "stressed" or "tired").

[0423] Step 5:

[0424] The server suggests appropriate meals based on the health and emotional state data. As input, it uses keywords indicating health abnormalities and the emotional state data and sends a request to the meal suggestion API (e.g., https: / / mydeliveryapi.example.com). The output is a suggested meal menu.

[0425] Step 6:

[0426] The device notifies the user of the meal suggestions sent from the server. As input, it receives the suggestion data from the server and displays a message to the user saying, "We will suggest nutritious meals that suit your condition." The output is a notification message that the user can see.

[0427] This allows users to receive appropriate health support.

[0428] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0429] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0430] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0431] [Second embodiment]

[0432] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0433] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0434] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0435] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0436] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0437] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0438] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0439] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0440] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0441] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0442] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0443] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0444] As an embodiment of the present invention, the system program is designed as follows.

[0445] Overall overview

[0446] This automated system collects voice data and search history data from users' smart devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of terminals, servers, and users.

[0447] System Configuration and Operation

[0448] Data collection

[0449] The device collects voice input and search history every time a user speaks to or searches on the smart device. For example, if a user says "my tooth hurts" to their smartphone, the device records the voice data. At the same time, if the user searches for keywords such as "toothache causes" on the browser, the device also collects that history data.

[0450] Data Conversion and Transmission

[0451] The collected voice data is converted into text data using the device's voice recognition function, and this converted text data and search history data are then sent to the server.

[0452] Data analysis

[0453] The server analyzes the submitted text data and search history data. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "toothache" or "tooth decay" are recognized, the server will determine that this is a type of health abnormality.

[0454] Identifying medical institutions and suggesting appointments

[0455] Based on the analysis results, the server takes into account the user's current location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[0456] User Notification

[0457] The device displays the appointment suggestion notification sent from the server to the user. For example, a message such as "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?" can be displayed. The user can choose to respond with "Yes" or "No" to this message.

[0458] Confirmation of reservation and notification of details

[0459] If the user answers "yes," the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) are notified to the terminal.

[0460] Reminder function

[0461] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[0462] Specific examples

[0463] 1. The user says to their smartphone, "I've been having stomach aches lately."

[0464] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[0465] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[0466] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[0467] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[0468] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[0469] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[0470] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[0471] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0472] Through this series of processes, users can quickly recognize any health abnormalities and ensure that they are referred to an appropriate medical institution.

[0473] The processing flow will be explained below.

[0474] Step 1:

[0475] The user speaks a health-related phrase (e.g., "My tooth hurts") into their smart device.

[0476] Step 2:

[0477] The device uses voice recognition to capture the user's voice data and convert it into text data.

[0478] Step 3:

[0479] The device retrieves the user's search history data from the browser.

[0480] Step 4:

[0481] The terminal transmits the converted text data and the acquired search history data to the server.

[0482] Step 5:

[0483] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[0484] Step 6:

[0485] The server searches for an appropriate medical institution using the user's location information based on the extracted keywords.

[0486] Step 7:

[0487] The server obtains the available appointment dates and times for the medical institutions found by the search.

[0488] Step 8:

[0489] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[0490] Step 9:

[0491] The device displays a notification message from the server to the user (e.g., "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?").

[0492] Step 10:

[0493] The user responds to the notification with a "yes" or "no" answer.

[0494] Step 11:

[0495] The terminal sends the user's response results to the server.

[0496] Step 12:

[0497] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[0498] Step 13:

[0499] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[0500] Step 14:

[0501] The terminal displays a reservation confirmation message from the server to the user.

[0502] Step 15:

[0503] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[0504] Step 16:

[0505] The device displays a reminder notification from the server to the user.

[0506] Example 1

[0507] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0508] In modern society, there are an increasing number of situations where users need to quickly understand their own health status and seek medical attention at an appropriate medical institution. However, users often need a great deal of time and effort to collect medical information, and there is a lack of appropriate systems for selecting and booking medical institutions. In particular, the time required for early detection of health abnormalities and selecting an appropriate medical institution is an issue.

[0509] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0510] In this invention, the server includes means for collecting voice data and search history data from the user's electronic device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicative of health abnormalities, means for identifying an appropriate medical institution based on the keywords and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's electronic device, means for cooperating with the medical institution's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's electronic device of detailed information about the appointment, and means for generating and sending a reminder notification when the appointment date and time approaches. This allows the user to quickly understand their own health abnormalities and ensure that they visit an appropriate medical institution.

[0511] "User" refers to an individual or organization that uses an electronic device.

[0512] "Electronic terminal" refers to devices with communication capabilities, such as smart devices, personal computers, and tablets.

[0513] "Audio data" refers to digital data acquired as audio input.

[0514] "Search History Data" refers to data that includes the history of searches you perform on your internet browser or applications.

[0515] "Text data" refers to data in which voice data has been converted into a string of characters.

[0516] "Health abnormalities" refers to any abnormal or unwell health condition experienced by the User.

[0517] "Keywords" refer to specific words or phrases related to a health condition.

[0518] "Medical institution" refers to a facility that provides medical services, such as a doctor's office, hospital, or clinic.

[0519] "Available appointment dates" refers to dates and times when a medical institution can accept appointments for medical treatment.

[0520] "Notification" means the sending of a message or alert to a User's electronic device.

[0521] "Reminder notification" refers to notifying the user again when the reservation deadline approaches.

[0522] "Server" refers to a computer system that stores, manages, analyzes, and communicates data over a network.

[0523] As an embodiment of the present invention, the system program is implemented as follows.

[0524] System Overview

[0525] This automated system collects voice data and search history data from users' electronic devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of three elements: the device, the server, and the user.

[0526] Data collection and transformation

[0527] The device collects voice input and search history every time a user speaks to the device or searches. For example, if a user says to the device, "I've had a stomach ache recently," the device records the voice data. Similarly, if a user searches for "causes of stomach ache" on a browser, the device also collects that history data.

[0528] The collected voice data is converted into text data using the device's voice recognition function (e.g., a cloud-based voice recognition API). Specifically, the Google Cloud Speech-to-Text API can be used. This converted text data and search history data are then sent to a server.

[0529] Data analysis

[0530] The server analyzes the text data and search history data sent to it. Natural language processing (NLP) technology is used for the analysis. For example, keywords indicating health abnormalities are extracted using the Google Cloud Natural Language API. Specifically, keywords such as "my stomach hurts" and "causes of stomach pain" are extracted from the text data and search history data, and the presence or absence of health abnormalities is determined based on this.

[0531] Identifying medical institutions and suggesting appointments

[0532] Based on the analysis results, the server takes into account the user's current location information (e.g., GPS data) and searches for suitable nearby medical institutions. It then obtains available appointment dates and times for each medical institution and creates an appointment proposal. The appointment proposal includes a specific message such as "Would you like to make an appointment at a nearby internal medicine clinic?" The proposal is then sent to the device as a notification.

[0533] User notification and booking confirmation

[0534] The terminal displays the appointment suggestion notification sent from the server to the user. If the user responds "Yes" to this message, the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the terminal is notified of the details (date, time, location, name of the medical institution, etc.).

[0535] Reminder function

[0536] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[0537] Specific examples

[0538] 1. The user speaks to an electronic device saying, "I've been having stomach aches lately."

[0539] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[0540] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[0541] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[0542] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[0543] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[0544] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[0545] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[0546] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0547] Prompt Sentence Examples

[0548] Example prompt sentence:

[0549] Please describe a system that detects health abnormalities and schedules appropriate medical appointments by having users talk to their smartphones or look at their search history. Please include the following information: the hardware and software used, what data is collected and analyzed, how it is collected and analyzed, how users are notified, and specific examples.

[0550] Through the above-described procedures, the present invention assists the user in managing his or her health and ensures prompt and appropriate medical treatment.

[0551] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0552] Step 1: Data collection

[0553] A user speaks to an electronic device or performs a search. The user's voice data and search history data are obtained as input. The device records the voice data when the user speaks to the electronic device, such as "I've had a stomachache recently," and collects search history data for "causes of stomach pain" on the browser. The recorded voice data and search history data are generated as output.

[0554] Step 2: Data conversion

[0555] The device converts the collected voice data into text data using a speech recognition function (for example, Google Cloud Speech-to-Text API). The recorded voice data is given as input. As a result of the conversion, the text data "I've had a stomach ache recently" is output. Furthermore, the search history data is used in its original form.

[0556] Step 3: Sending data

[0557] The terminal transmits the converted text data and search history data to the server. As input, the text data and search history data are provided. These data are encrypted and transmitted to the server. As output, the text data and search history data received by the server are obtained.

[0558] Step 4: Data analysis

[0559] The server analyzes the received text data and search history data to extract keywords that indicate health abnormalities. As input, the received text data "I've had a stomach ache recently" and search history data "Causes of stomach pain" are given. The server uses the Google Cloud Natural Language API to extract keywords such as "my stomach hurts" and "stomach pain" from these data. The extracted keywords are generated as output.

[0560] Step 5: Identify medical facilities

[0561] The server determines health abnormalities based on the extracted keywords and identifies appropriate medical institutions. The extracted keywords and the user's current location information (e.g., GPS data) are given as input. The server searches for nearby medical institutions and obtains available appointment dates and times. The output is the medical institution and available appointment date and time information.

[0562] Step 6: Creating a booking proposal

[0563] The server creates an appointment suggestion based on the retrieved medical institution and available appointment date and time. Medical institution information and available appointment date and time information are given as input. The server creates an appointment suggestion such as "Would you like to make an appointment at a nearby internal medicine clinic?". As output, an appointment suggestion notification is generated.

[0564] Step 7: Send booking proposal notification

[0565] The server sends the created reservation proposal notification to the terminal. The reservation proposal notification is given as input. The server sends the reservation proposal notification to the user's terminal. The reservation proposal notification received by the terminal is obtained as output.

[0566] Step 8: Receiving user response

[0567] The user responds to the reservation suggestion notification displayed on the terminal. As input, the user's response "yes" or "no" is obtained. For example, the user responds "yes." As output, the user's response is generated.

[0568] Step 9: Confirm your booking

[0569] The server receives the user's response and confirms the appointment by linking with the medical institution's appointment system. The user's response and the medical institution's appointment information are given as input. The server confirms the appointment for the specified date and time through the medical institution's online appointment system. The detailed appointment information is generated as output.

[0570] Step 10: Send reservation details notification

[0571] The server sends the details of the confirmed reservation to the terminal. As input, the reservation details are given. The server sends the reservation details to the user's terminal. As output, a reservation details notification received by the terminal is obtained.

[0572] Step 11: Generate a reminder notification

[0573] The server generates a reminder notification when the reservation date and time is approaching. Reservation date and time information is given as input. The server generates the reminder notification. The generated reminder notification is obtained as output.

[0574] Step 12: Send reminders

[0575] The server sends the generated remind notification to the terminal. The remind notification is given as input. The server sends the remind notification to the user's terminal. The output is the remind notification received by the terminal.

[0576] (Application example 1)

[0577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] Conventional systems lack the ability to quickly analyze a user's health status and recommend appropriate medical institutions and health-related products. As a result, users have no means to respond immediately to their health status and spend a lot of time and effort finding appropriate medical services and health products. The present invention aims to solve this problem by providing a system that allows users to quickly recognize health abnormalities and receive recommendations for appropriate medical institutions and health-related products.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0580] In this invention, the server includes means for collecting voice data and search history data from the user's computer terminal, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting words or phrases indicating health abnormalities, means for identifying an appropriate medical facility based on the words or phrases and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's computer terminal, means for coordinating with the medical facility's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's computer terminal of details of the appointment, and means for analyzing the user's health condition data and suggesting health-related products. This allows the user to quickly respond to their health condition and receive appointments at appropriate medical institutions and suggestions for health-related products.

[0581] A "user" is a person who uses this system and provides voice data and search history data.

[0582] "Computer terminal" means a device used by a user, and is a digital device including a smartphone, tablet, PC, smart glasses, etc.

[0583] "Voice data" refers to voice information uttered by a user into a computer terminal and analyzed using voice recognition technology.

[0584] "Search history data" refers to the history of searches a user has conducted on the Internet, and is used to infer the user's interests and health status.

[0585] "Character data" is information obtained by analyzing voice data and converting it into text.

[0586] "Terms" are keywords and phrases extracted from voice data and search history data, and are used to identify health abnormalities and suggest products.

[0587] A "medical facility" is a facility where a person can receive medical treatment from a doctor, and refers to general forms of medical institutions, including hospitals and clinics.

[0588] The "available appointment dates and times" are specific dates and times when a medical facility can accept a user's appointment.

[0589] "Reservation proposal" refers to the content of a proposal for a medical facility reservation that the server sends to the user based on the analysis results.

[0590] "Health Status Data" refers to information related to a user's health that is inferred based on goad cues obtained from voice data and search history data.

[0591] "Health-related products" are products suggested based on the user's health condition, including supplements, medicines, health foods, etc.

[0592] Overall overview

[0593] As an embodiment of the present invention, the system program is designed as follows.

[0594] System Configuration and Operation

[0595] Data collection

[0596] First, the user inputs their health condition into a computer terminal by voice. For example, the user might say, "I've been feeling tired lately." This voice data is converted into text data by voice recognition software (e.g., Google Cloud Speech-to-Text API) running on the computer terminal. At the same time, if the user searches online for something like "ways to recover from fatigue," that search history data is also collected.

[0597] Data analysis

[0598] The server receives the converted text data and search history data and analyzes it using natural language processing (NLP) technology (e.g., Transformers with the BERT model). This analysis extracts words that indicate health abnormalities. For example, the analysis results may include words such as "fatigue" and "rest."

[0599] Identifying and recommending medical facilities and products

[0600] Based on the analysis results, the server takes into account the user's location information and identifies appropriate medical facilities and health-related products, such as nearby internal medicine clinics or supplements effective for fatigue recovery. This information is then sent to the computer terminal along with available appointment times and detailed product information.

[0601] User Notification and Response

[0602] The user receives information about the proposed medical facility and product on the computer terminal. For example, a message such as "Would you like to purchase a fatigue recovery supplement?" or "I have made an appointment with a nearby internal medicine clinic. Would you like to confirm the appointment?" is displayed. If the user accepts the appointment suggestion, the server connects with the medical facility's reservation system to confirm the appointment. If the user responds to the product suggestion, the purchase procedure is carried out.

[0603] Reservation details and reminders

[0604] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. For example, it may say, "Your reservation has been made at the internal medicine clinic on a certain date at a certain time." In addition, when the reservation date and time approaches, a reminder notification is generated and sent again to the computer terminal to notify the user.

[0605] Specific examples

[0606] A user says to their smartphone, "I have a headache."

[0607] 1. Speech recognition software converts the speech into the text "I have a headache."

[0608] 2. The converted character data is sent to the server.

[0609] 3. The server analyzes the text data and search history data to extract words that indicate health abnormalities, such as "headache."

[0610] 4. Based on the analysis results, medical facilities and health products related to headaches are identified, and detailed information is sent from the server to the smartphone.

[0611] 5. The user responds to prompts such as "Would you like to make an appointment with a local doctor?" or "Would you like to buy some headache medicine?"

[0612] 6. Once the user approves the reservation, the server connects to the medical facility's reservation system to confirm the reservation.

[0613] 7. The server will notify your smartphone of the reservation details.

[0614] 8. When your appointment time approaches, a reminder notification will be sent to your smartphone.

[0615] Prompt Sentence Examples

[0616] "Please generate an application program that analyzes the user's health-related voice data and suggests appropriate medical institutions and health products. For example, if a user says, 'I have a headache,' convert the voice into text, analyze that data, and suggest related products."

[0617] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0618] Step 1:

[0619] A user inputs voice into a computer terminal. At this time, the computer terminal uses voice recognition software (e.g., Google Cloud Speech-to-Text API) to collect the voice data and convert it into text data. The input is the user's voice data, and the output is the converted text data.

[0620] Step 2:

[0621] The terminal sends the converted text data to the server. At the same time, the user's recent search history data is also sent to the server. The input is the text data and the search history data, and the output is the data sent to the server.

[0622] Step 3:

[0623] The server analyzes the received text data and search history data using natural language processing (NLP) techniques (e.g., Transformers with the BERT model). This analysis extracts phrases that indicate health abnormalities. The input is the text data and search history data, and the output is the extracted health abnormality phrases.

[0624] Step 4:

[0625] The server identifies appropriate medical facilities based on the analysis results and takes into account the user's location information. At the same time, it searches the database for health-related products related to the health abnormality. The input is the health abnormality phrase and location information, and the output is a list of identified medical facilities and health-related products.

[0626] Step 5:

[0627] The server creates reservation proposals and product proposals, including available appointment dates and times and detailed product information, and notifies the user of these proposals. The input is detailed information about medical facilities and products, and the output is a notification to the user.

[0628] Step 6:

[0629] The user checks the proposed content on the computer terminal and responds, for example, by answering "yes" or "no" to messages such as "Would you like to make a reservation?" or "Would you like to purchase a product?" The input at this step is the user response, and the output is the user's choice.

[0630] Step 7:

[0631] If the user accepts the reservation proposal, the server will confirm the reservation in cooperation with the medical facility's reservation system. If the user accepts the product proposal, the server will proceed with the purchase procedure. The input is the user's selection, and the output is the confirmed reservation and a notification of purchase completion.

[0632] Step 8:

[0633] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. Furthermore, when the reservation date and time approaches, a reminder notification is generated and sent to the user's computer terminal. The input is the confirmed reservation information, and the output is the reservation details notification and the reminder notification.

[0634] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0635] In one embodiment of the present invention, we design an automated system that collects voice data and search history data from a user's smart device, analyzes this data to detect health abnormalities, recommends appropriate medical institutions, and schedules appointments. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to perform additional analysis and judgment based on the user's emotional state.

[0636] Overall overview

[0637] This system consists of a terminal, a server, and a user. The terminal is a smart device that collects voice data and search history data. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. An emotion engine has also been added to evaluate the user's emotional state and complement the health abnormality analysis results.

[0638] System Configuration and Operation

[0639] Data collection

[0640] The device collects voice data and search history data every time the user speaks to the smart device or searches the internet. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device will record the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device will also collect that search history data.

[0641] Data Conversion and Transmission

[0642] The collected voice data is converted into text data using the device's voice recognition function, and the converted text data and search history data are sent to the server.

[0643] Data analysis

[0644] The server analyzes the text data and search history data sent to it. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[0645] Emotion recognition and evaluation

[0646] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine evaluates the user's emotional state based on their vocabulary, tone of voice, search results, etc. For example, if a user says, "I'm having a really hard time sleeping," the engine detects their feelings of confusion and anxiety.

[0647] Identifying medical institutions and suggesting appointments

[0648] Based on the analysis results and emotion evaluation results, the server searches for suitable nearby medical institutions, taking into account the user's location information. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[0649] User Notification

[0650] The device displays a notification of the appointment suggestion sent from the server to the user. For example, a message such as "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?" can be displayed. The user can respond to this message with "Yes" or "No."

[0651] Confirmation of reservation and notification of details

[0652] If the user answers "yes," the server will connect to the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) will be notified to the terminal.

[0653] Reminder function

[0654] When the appointment time approaches, the server generates a reminder notification and sends it to the device, which displays the reminder notification to the user to remind them not to forget to come to the appointment.

[0655] Specific examples

[0656] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[0657] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I haven't been sleeping well lately," to the server.

[0658] 3. At the same time, the user's browser search history for "insomnia relief" is also sent to the server.

[0659] 4. The server analyzes keywords such as "can't sleep well" and "insomnia" and also evaluates the user's emotional state.

[0660] 5. Based on the analysis results, search for nearby sleep clinics and obtain available appointment dates and times.

[0661] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[0662] 7. If the user responds "yes," the server confirms the appointment with the designated sleep clinic.

[0663] 8. The terminal displays detailed information such as "An appointment has been made for the sleep clinic at XX / XX / XX at XX time."

[0664] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0665] The present invention, which combines emotion engines in this way, enables quick responses to the user's health condition, selection of an appropriate medical institution, and support for consultation.

[0666] The processing flow will be explained below.

[0667] Step 1:

[0668] The user speaks a health-related phrase into their smart device (e.g., "I haven't been sleeping well lately").

[0669] Step 2:

[0670] The device uses voice recognition to capture the user's voice data and convert it into text data.

[0671] Step 3:

[0672] The device retrieves the user's search history data from the browser.

[0673] Step 4:

[0674] The terminal transmits the converted text data and the acquired search history data to the server.

[0675] Step 5:

[0676] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[0677] Step 6:

[0678] The server uses an emotion engine to recognize the user's emotion from the voice data and text data and evaluates the user's emotional state.

[0679] Step 7:

[0680] The server evaluates the user's health abnormalities and determines the risk based on the extracted keywords and sentiment analysis results.

[0681] Step 8:

[0682] The server references the user's location information and searches for appropriate medical facilities nearby.

[0683] Step 9:

[0684] The server obtains the available appointment dates and times for the medical institutions found by the search.

[0685] Step 10:

[0686] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[0687] Step 11:

[0688] The device displays a notification message from the server to the user (e.g., "You haven't been sleeping well lately. Would you like to make an appointment with a nearby sleep clinic?").

[0689] Step 12:

[0690] The user responds to the notification with a "yes" or "no" answer.

[0691] Step 13:

[0692] The terminal sends the user's response results to the server.

[0693] Step 14:

[0694] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[0695] Step 15:

[0696] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[0697] Step 16:

[0698] The terminal displays a reservation confirmation message from the server to the user.

[0699] Step 17:

[0700] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[0701] Step 18:

[0702] The device displays a reminder notification from the server to the user.

[0703] Specifically, when a user says, "I haven't been sleeping well lately," the device converts the voice data into text and sends it to the server. The server analyzes the text data, extracts keywords such as "can't sleep" and "insomnia," and uses an emotion engine to evaluate the user's feelings of anxiety and confusion. Based on this, the device searches for appropriate medical institutions and their available appointment dates and times, and makes a reservation suggestion. If the user responds "yes," the reservation is confirmed. The reservation details and a reminder notification are then sent to the user.

[0704] Example 2

[0705] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] In modern society, it is extremely important for users to take prompt and appropriate measures regarding their health. However, the process of users recognizing their own health abnormalities and finding an appropriate medical institution is complicated and time-consuming. Furthermore, a user's emotional state and stress level also have a significant impact on their health, but there is a lack of responses that take these into consideration. Therefore, there is a need for a system that efficiently utilizes a user's voice data and search history data to detect health abnormalities, identify appropriate medical institutions, and assist in making appointments, while also taking their emotional state into consideration.

[0707] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting voice data and digital internet information from the user's information processing device; means for converting the voice data into text data; means for analyzing the text data and digital internet information and extracting an identifier indicating a health abnormality; means for identifying an appropriate medical facility and obtaining an available appointment date and time based on the identifier; means for sending a notification including an appointment proposal to the user's information processing device; means for confirming the appointment in cooperation with the medical facility's appointment system if the user accepts the appointment proposal; means for notifying the user's information processing device of details of the appointment; and means including an emotion engine that recognizes the user's emotional state and performs complementary analysis based on the analysis results. This allows the user to respond to their health condition quickly and appropriately, and accurate detection of health abnormalities and recommendations for medical institutions are realized taking the user's emotional state into consideration.

[0708] "User's information processing device" refers to an electronic device used by a user, such as a smartphone, tablet, or personal computer, that has the function of collecting and transmitting voice data and digital internet information.

[0709] "Voice data" refers to data that is recorded in digital format via a microphone of what a user says to an information processing device.

[0710] "Digital Internet information" refers to historical data such as the search history a user has conducted on the Internet and the web pages they have accessed.

[0711] The "means for converting into text data" refers to a speech recognition technology or system for converting voice data into text format.

[0712] "Identifiers" are keywords or phrases extracted from the data being analyzed that are used to indicate specific health abnormalities.

[0713] "Medical facilities" are medical institutions such as hospitals and clinics that provide medical treatment according to the user's health condition.

[0714] An "emotion engine" is a technology or system that recognizes a user's emotional state based on their voice data and text data, and provides analysis results according to their emotions.

[0715] A "reservation system" is a system used by medical facilities to manage reservations, and has functions for accepting, managing, and confirming reservations.

[0716] A "notification" is a message sent from the server to the user's information processing device, and includes information such as a medical facility reservation suggestion, detailed reservation information, and a reminder.

[0717] "Emotional state" refers to the psychological state analyzed from the user's voice and text data, and indicates emotions such as joy, sadness, anger, and anxiety.

[0718] "Complementary analytics" refers to additional or detailed analytics based on a user's emotional state that can be used to make more accurate decisions or recommendations.

[0719] As an embodiment of the present invention, we have designed an automated system that collects voice data and digital internet information from a user's information processing device, analyzes this data to detect health abnormalities, recommends appropriate medical facilities, and makes appointments. This system also incorporates an emotion engine that recognizes the user's emotional state, allowing it to perform additional analysis and judgment based on the user's emotional state.

[0720] Overall overview

[0721] This system consists of the user's information processing device (smart device), a server, and the user's components. The user's information processing device collects voice data and digital internet information. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. In addition, an emotion engine has been added to evaluate the user's emotional state and complement the analysis results of health abnormalities.

[0722] Hardware and software used

[0723] Information processing equipment: smart devices (smartphones, tablets, etc.)

[0724] Server: Dedicated server for data analysis

[0725] Speech recognition technology: Google Speech-to-Text API, etc.

[0726] NLP engines: spaCy and NLTK built in Python

[0727] Emotion recognition engine: Uses Praat and openSMILE as voice analysis tools

[0728] Geographic information service: Google Maps API

[0729] Push notification service: Firebase Cloud Messaging

[0730] Natural language description of the process

[0731] The device collects voice data when the user speaks to the smart device and the search history data they perform on the Internet. For example, if a user says, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device collects the search history.

[0732] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are then sent to a server. The server then uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[0733] The server uses an emotion engine to recognize the user's emotional state from text and voice data. For example, if someone says, "I'm having a really hard time sleeping," the server can detect feelings of confusion and anxiety. This emotion assessment is used to complement the analysis results.

[0734] Based on the analysis results and emotion evaluation results, the server takes into account the user's location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates an appointment suggestion. The device displays the appointment suggestion notification sent from the server to the user. For example, a message may appear saying, "You seem to be having trouble sleeping. Would you like to make an appointment at a nearby sleep clinic?" The user can respond to this message with "Yes" or "No."

[0735] If the user responds "Yes," the server will link with the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, detailed information (date, time, location, name of medical institution, etc.) will be notified to the device. When the appointment date and time approaches, the server will generate a reminder notification and send it to the device. The device will display this reminder notification to the user to remind them not to forget to visit.

[0736] Specific examples

[0737] Example 1:

[0738] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[0739] 2. The device converts the speech into text using a speech recognition function (for example, Google Speech-to-Text API) and sends the text data, such as "I haven't been sleeping well lately," to the server.

[0740] 3. At the same time, the user's browser search history data for "insomnia relief" is sent to the server.

[0741] 4. The server uses a Python NLP engine (e.g., spaCy) to analyze keywords such as "can't sleep well" and "insomnia" and also evaluate the user's emotional state.

[0742] 5. Based on the analysis results, use the Google Maps API to search for nearby sleep clinics and obtain available appointment dates and times.

[0743] 6. Using Firebase Cloud Messaging, display a notification on the device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[0744] 7. If the user answers "Yes," the server calls the medical institution's reservation API to confirm the appointment with the specified sleep clinic.

[0745] 8. The device will display detailed information such as "Your appointment has been made at the sleep clinic on XX / XX / XX at XX time." A reminder notification will also be scheduled.

[0746] Prompt Sentence Examples

[0747] Please explain the process of suggesting an appointment for a sleep clinic to a user who says, "I haven't been sleeping well lately."

[0748] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0749] Step 1:

[0750] The device collects voice data when the user speaks to the smart device and the search history data of the user's internet searches. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "insomnia relief," the device collects the search history.

[0751] Input: User voice data, internet search history data

[0752] Output: Recorded voice data, collected search history data

[0753] Step 2:

[0754] The collected voice data is converted into text data using the device's voice recognition function. For example, the Google Speech-to-Text API is used to convert the speech "I haven't been sleeping well lately" into text. The converted text data and search history data are then sent to the server.

[0755] Input: Recorded audio data

[0756] Output: Text data converted to character data, search history data

[0757] Step 3:

[0758] The server analyzes the text data and search history data sent to it. It uses natural language processing (NLP) technology, such as a Python NLP engine (spaCy or NLTK), to extract keywords that indicate health abnormalities. For example, it could extract keywords such as "not sleeping well" or "insomnia" and determine that the user has sleep problems.

[0759] Input: Text data converted to character data, search history data

[0760] Output: Extracted keywords related to health abnormalities

[0761] Step 4:

[0762] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine uses speech analysis tools (Praat and openSMILE) to analyze the intonation, speed, and volume of the voice data and extract emotions from the text data. For example, if a user says, "I'm having a really hard time sleeping," the emotion engine will detect confusion and anxiety.

[0763] Input: Text data converted to character data, audio data

[0764] Output: Data about the user's emotional state

[0765] Step 5:

[0766] Based on the analysis results and emotion evaluation results, the server searches for appropriate nearby medical institutions, taking into account the user's location information. For example, it uses the Google Maps API to search for nearby sleep clinics based on the user's location information and obtains available appointment dates and times at each medical institution.

[0767] Input: location information, data analysis results, emotion evaluation results

[0768] Output: List of suitable medical institutions, available appointment dates and times

[0769] Step 6:

[0770] The device displays the suggestions for available medical institutions sent from the server as a notification to the user. For example, it uses Firebase Cloud Messaging to notify the user with a message such as, "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?"

[0771] Input: List of suitable medical institutions, available appointment dates and times

[0772] Output: Notification of booking proposal to user

[0773] Step 7:

[0774] If the user accepts the reservation proposal, the server cooperates with the medical institution's reservation system to confirm the reservation at the specified date and time, for example, through the medical institution's reservation API.

[0775] Input: User response

[0776] Output: Confirmed reservation information

[0777] Step 8:

[0778] After the reservation is successfully made, the details (date, time, location, name of medical institution, etc.) are notified to the device. In addition, when the reservation date and time approaches, the server generates a reminder notification and sends it to the device. The device displays this reminder notification to the user.

[0779] Input: Confirmed reservation information, schedule information for reminder notifications

[0780] Output: Reservation details notification, reminder notification

[0781] (Application example 2)

[0782] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0783] Conventional health management systems and food delivery services have been unable to effectively utilize users' voice data and search history data to detect health abnormalities or recommend appropriate medical institutions. Furthermore, they have not made meal suggestions that take into account the user's emotional state, making it difficult for users to select meals based on their own health condition and emotions. This has led to the issue of users being unable to receive appropriate support to maintain and improve their health.

[0784] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0785] In this invention, the server includes means for collecting voice data and search history data from the user's information processing device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicating health abnormalities, means for identifying appropriate medical institutions based on the keywords and obtaining available appointment dates and times, means for suggesting appropriate meals based on the user's health abnormalities and emotional state, and means for transmitting the contents of the suggested meals to the information processing device. This allows the user to receive meal suggestions based on their health and emotional state, and also enables them to quickly make appointments at necessary medical institutions.

[0786] An "information processing device" is an electronic device used by a user, such as a smartphone, tablet, or computer, and is a device for collecting and analyzing voice data and search history data.

[0787] "Voice data" refers to digital data that records what a user says to an information processing device.

[0788] "Search history data" refers to data that records the history of a user's internet searches.

[0789] "Text data" refers to data obtained by converting voice data into character information.

[0790] "Keywords" are important words that indicate health abnormalities or emotional states, extracted from text data and search history data.

[0791] A "medical institution" is an institution that provides medical services, such as a hospital or clinic.

[0792] An "emotion engine" is software that analyzes a user's emotional state from their voice and text data.

[0793] "Meal suggestions" are information that recommends optimal meals based on the user's health and emotional state.

[0794] A "notification" is a message or an alert sent to an information processing device.

[0795] A "reservation system" is a system for managing and processing reservations at medical institutions.

[0796] As an embodiment of the present invention, a system applied to a food delivery service consisting of a user, a terminal, and a server will be described. The operation of each element and the overall flow will be described in detail below.

[0797] Configuration and Operation

[0798] Data collection

[0799] The device collects voice data entered by the user using the voice recognition function. For example, if the user says, "I've been feeling tired lately," the device records the voice and saves it as voice data. At the same time, it collects the keywords the user searched for (e.g., "how to relieve fatigue") as search history data.

[0800] Data Conversion and Transmission

[0801] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are sent to the server. For voice recognition, the speech_recognition library, for example, is used.

[0802] Data analysis

[0803] The server uses natural language processing (NLP) technology to analyze the submitted text data and search history data, extracting keywords that indicate health abnormalities (e.g., "fatigue" and "easily tired"). The analysis uses a health API (e.g., https: / / myhealthapi.example.com).

[0804] Emotion recognition and evaluation

[0805] The server identifies the user's emotional state using an emotion engine. This emotion engine evaluates the user's emotion from text data and voice tone. For emotion analysis, an emotion analysis API (e.g., https: / / myfeelingsapi.example.com) is used.

[0806] Meal suggestions

[0807] Based on the analyzed health and emotion data, the server suggests appropriate meals using a meal suggestion API (e.g., https: / / mydeliveryapi.example.com).

[0808] User Notification

[0809] The device will notify the user of the meal suggestions sent from the server, for example, by displaying a message saying, "We will suggest nutritious meals that suit your condition."

[0810] Specific examples

[0811] For example, if a user says, "I've been feeling a bit tired lately," the system analyzes the voice data and determines that the user is tired. Next, it performs emotion analysis to evaluate the user's emotions. Based on this, it suggests nutritious meals (e.g., vitamin-rich meals) and notifies the information processing device so that the user can immediately order the meals.

[0812] Prompt Sentence Examples

[0813] "User voice data: 'I've been feeling a bit tired lately.' Search history: 'How to relieve fatigue.' Analyze this data to suggest nutritious meals that are suitable for the user."

[0814] In this way, users can receive optimal support based on their health and emotional state.

[0815] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0816] Step 1:

[0817] The device collects voice data entered by the user using the voice recognition function. The input requires the user's voice, such as saying, "I've been feeling tired lately." The voice data is stored as digital data on the device. In addition, if the user enters search keywords such as "how to relieve fatigue," the device also collects search history data.

[0818] Step 2:

[0819] The device converts the collected voice data into text data using a voice recognition function. Digital voice data is used as input, and the conversion is performed using, for example, the speech_recognition library. The output is string data that can be analyzed from the voice data. At the same time, the converted text data and search history data are sent to the server.

[0820] Step 3:

[0821] The server analyzes the submitted text data and search history data using natural language processing (NLP) techniques. The input is the text data and search history data, and keywords (e.g., "fatigue" and "easily tired") are extracted using a health API (e.g., https: / / myhealthapi.example.com). The extracted keywords are obtained as output.

[0822] Step 4:

[0823] The server uses an emotion engine to analyze the user's emotions from the text data. It uses the text data obtained in step 3 as input and utilizes an emotion analysis API (e.g., https: / / myfeelingsapi.example.com). It outputs the user's emotional state (e.g., "stressed" or "tired").

[0824] Step 5:

[0825] The server suggests appropriate meals based on the health and emotional state data. As input, it uses keywords indicating health abnormalities and the emotional state data and sends a request to the meal suggestion API (e.g., https: / / mydeliveryapi.example.com). The output is a suggested meal menu.

[0826] Step 6:

[0827] The device notifies the user of the meal suggestions sent from the server. As input, it receives the suggestion data from the server and displays a message to the user saying, "We will suggest nutritious meals that suit your condition." The output is a notification message that the user can see.

[0828] This allows users to receive appropriate health support.

[0829] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0830] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0831] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0832] [Third embodiment]

[0833] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0834] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0835] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0836] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0837] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0838] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0839] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0840] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0841] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0842] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0843] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0844] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0845] As an embodiment of the present invention, the system program is designed as follows.

[0846] Overall overview

[0847] This automated system collects voice data and search history data from users' smart devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of terminals, servers, and users.

[0848] System Configuration and Operation

[0849] Data collection

[0850] The device collects voice input and search history every time a user speaks to or searches on the smart device. For example, if a user says "my tooth hurts" to their smartphone, the device records the voice data. At the same time, if the user searches for keywords such as "toothache causes" on the browser, the device also collects that history data.

[0851] Data Conversion and Transmission

[0852] The collected voice data is converted into text data using the device's voice recognition function, and this converted text data and search history data are then sent to the server.

[0853] Data analysis

[0854] The server analyzes the submitted text data and search history data. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "toothache" or "tooth decay" are recognized, the server will determine that this is a type of health abnormality.

[0855] Identifying medical institutions and suggesting appointments

[0856] Based on the analysis results, the server takes into account the user's current location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[0857] User Notification

[0858] The device displays the appointment suggestion notification sent from the server to the user. For example, a message such as "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?" can be displayed. The user can choose to respond with "Yes" or "No" to this message.

[0859] Confirmation of reservation and notification of details

[0860] If the user answers "yes," the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) are notified to the terminal.

[0861] Reminder function

[0862] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[0863] Specific examples

[0864] 1. The user says to their smartphone, "I've been having stomach aches lately."

[0865] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[0866] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[0867] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[0868] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[0869] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[0870] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[0871] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[0872] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0873] Through this series of processes, users can quickly recognize any health abnormalities and ensure that they are referred to an appropriate medical institution.

[0874] The processing flow will be explained below.

[0875] Step 1:

[0876] The user speaks a health-related phrase (e.g., "My tooth hurts") into their smart device.

[0877] Step 2:

[0878] The device uses voice recognition to capture the user's voice data and convert it into text data.

[0879] Step 3:

[0880] The device retrieves the user's search history data from the browser.

[0881] Step 4:

[0882] The terminal transmits the converted text data and the acquired search history data to the server.

[0883] Step 5:

[0884] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[0885] Step 6:

[0886] The server searches for an appropriate medical institution using the user's location information based on the extracted keywords.

[0887] Step 7:

[0888] The server obtains the available appointment dates and times for the medical institutions found by the search.

[0889] Step 8:

[0890] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[0891] Step 9:

[0892] The device displays a notification message from the server to the user (e.g., "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?").

[0893] Step 10:

[0894] The user responds to the notification with a "yes" or "no" answer.

[0895] Step 11:

[0896] The terminal sends the user's response results to the server.

[0897] Step 12:

[0898] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[0899] Step 13:

[0900] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[0901] Step 14:

[0902] The terminal displays a reservation confirmation message from the server to the user.

[0903] Step 15:

[0904] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[0905] Step 16:

[0906] The device displays a reminder notification from the server to the user.

[0907] Example 1

[0908] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0909] In modern society, there are an increasing number of situations where users need to quickly understand their own health status and seek medical attention at an appropriate medical institution. However, users often need a great deal of time and effort to collect medical information, and there is a lack of appropriate systems for selecting and booking medical institutions. In particular, the time required for early detection of health abnormalities and selecting an appropriate medical institution is an issue.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0911] In this invention, the server includes means for collecting voice data and search history data from the user's electronic device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicative of health abnormalities, means for identifying an appropriate medical institution based on the keywords and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's electronic device, means for cooperating with the medical institution's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's electronic device of detailed information about the appointment, and means for generating and sending a reminder notification when the appointment date and time approaches. This allows the user to quickly understand their own health abnormalities and ensure that they visit an appropriate medical institution.

[0912] "User" refers to an individual or organization that uses an electronic device.

[0913] "Electronic terminal" refers to devices with communication capabilities, such as smart devices, personal computers, and tablets.

[0914] "Audio data" refers to digital data acquired as audio input.

[0915] "Search History Data" refers to data that includes the history of searches you perform on your internet browser or applications.

[0916] "Text data" refers to data in which voice data has been converted into a string of characters.

[0917] "Health abnormalities" refers to any abnormal or unwell health condition experienced by the User.

[0918] "Keywords" refer to specific words or phrases related to a health condition.

[0919] "Medical institution" refers to a facility that provides medical services, such as a doctor's office, hospital, or clinic.

[0920] "Available appointment dates" refers to dates and times when a medical institution can accept appointments for medical treatment.

[0921] "Notification" means the sending of a message or alert to a User's electronic device.

[0922] "Reminder notification" refers to notifying the user again when the reservation deadline approaches.

[0923] "Server" refers to a computer system that stores, manages, analyzes, and communicates data over a network.

[0924] As an embodiment of the present invention, the system program is implemented as follows.

[0925] System Overview

[0926] This automated system collects voice data and search history data from users' electronic devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of three elements: the device, the server, and the user.

[0927] Data collection and transformation

[0928] The device collects voice input and search history every time a user speaks to the device or searches. For example, if a user says to the device, "I've had a stomach ache recently," the device records the voice data. Similarly, if a user searches for "causes of stomach ache" on a browser, the device also collects that history data.

[0929] The collected voice data is converted into text data using the device's voice recognition function (e.g., a cloud-based voice recognition API). Specifically, the Google Cloud Speech-to-Text API can be used. This converted text data and search history data are then sent to a server.

[0930] Data analysis

[0931] The server analyzes the text data and search history data sent to it. Natural language processing (NLP) technology is used for the analysis. For example, keywords indicating health abnormalities are extracted using the Google Cloud Natural Language API. Specifically, keywords such as "my stomach hurts" and "causes of stomach pain" are extracted from the text data and search history data, and the presence or absence of health abnormalities is determined based on this.

[0932] Identifying medical institutions and suggesting appointments

[0933] Based on the analysis results, the server takes into account the user's current location information (e.g., GPS data) and searches for suitable nearby medical institutions. It then obtains available appointment dates and times for each medical institution and creates an appointment proposal. The appointment proposal includes a specific message such as "Would you like to make an appointment at a nearby internal medicine clinic?" The proposal is then sent to the device as a notification.

[0934] User notification and booking confirmation

[0935] The terminal displays the appointment suggestion notification sent from the server to the user. If the user responds "Yes" to this message, the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the terminal is notified of the details (date, time, location, name of the medical institution, etc.).

[0936] Reminder function

[0937] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[0938] Specific examples

[0939] 1. The user speaks to an electronic device saying, "I've been having stomach aches lately."

[0940] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[0941] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[0942] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[0943] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[0944] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[0945] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[0946] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[0947] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[0948] Prompt Sentence Examples

[0949] Example prompt sentence:

[0950] Please describe a system that detects health abnormalities and schedules appropriate medical appointments by having users talk to their smartphones or look at their search history. Please include the following information: the hardware and software used, what data is collected and analyzed, how it is collected and analyzed, how users are notified, and specific examples.

[0951] Through the above-described procedures, the present invention assists the user in managing his or her health and ensures prompt and appropriate medical treatment.

[0952] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0953] Step 1: Data collection

[0954] A user speaks to an electronic device or performs a search. The user's voice data and search history data are obtained as input. The device records the voice data when the user speaks to the electronic device, such as "I've had a stomachache recently," and collects search history data for "causes of stomach pain" on the browser. The recorded voice data and search history data are generated as output.

[0955] Step 2: Data conversion

[0956] The device converts the collected voice data into text data using a speech recognition function (for example, Google Cloud Speech-to-Text API). The recorded voice data is given as input. As a result of the conversion, the text data "I've had a stomach ache recently" is output. Furthermore, the search history data is used in its original form.

[0957] Step 3: Sending data

[0958] The terminal transmits the converted text data and search history data to the server. As input, the text data and search history data are provided. These data are encrypted and transmitted to the server. As output, the text data and search history data received by the server are obtained.

[0959] Step 4: Data analysis

[0960] The server analyzes the received text data and search history data to extract keywords that indicate health abnormalities. As input, the received text data "I've had a stomach ache recently" and search history data "Causes of stomach pain" are given. The server uses the Google Cloud Natural Language API to extract keywords such as "my stomach hurts" and "stomach pain" from these data. The extracted keywords are generated as output.

[0961] Step 5: Identify medical facilities

[0962] The server determines health abnormalities based on the extracted keywords and identifies appropriate medical institutions. The extracted keywords and the user's current location information (e.g., GPS data) are given as input. The server searches for nearby medical institutions and obtains available appointment dates and times. The output is the medical institution and available appointment date and time information.

[0963] Step 6: Creating a booking proposal

[0964] The server creates an appointment suggestion based on the retrieved medical institution and available appointment date and time. Medical institution information and available appointment date and time information are given as input. The server creates an appointment suggestion such as "Would you like to make an appointment at a nearby internal medicine clinic?". As output, an appointment suggestion notification is generated.

[0965] Step 7: Send booking proposal notification

[0966] The server sends the created reservation proposal notification to the terminal. The reservation proposal notification is given as input. The server sends the reservation proposal notification to the user's terminal. The reservation proposal notification received by the terminal is obtained as output.

[0967] Step 8: Receiving user response

[0968] The user responds to the reservation suggestion notification displayed on the terminal. As input, the user's response "yes" or "no" is obtained. For example, the user responds "yes." As output, the user's response is generated.

[0969] Step 9: Confirm your booking

[0970] The server receives the user's response and confirms the appointment by linking with the medical institution's appointment system. The user's response and the medical institution's appointment information are given as input. The server confirms the appointment for the specified date and time through the medical institution's online appointment system. The detailed appointment information is generated as output.

[0971] Step 10: Send reservation details notification

[0972] The server sends the details of the confirmed reservation to the terminal. As input, the reservation details are given. The server sends the reservation details to the user's terminal. As output, a reservation details notification received by the terminal is obtained.

[0973] Step 11: Generate a reminder notification

[0974] The server generates a reminder notification when the reservation date and time is approaching. Reservation date and time information is given as input. The server generates the reminder notification. The generated reminder notification is obtained as output.

[0975] Step 12: Send reminders

[0976] The server sends the generated remind notification to the terminal. The remind notification is given as input. The server sends the remind notification to the user's terminal. The output is the remind notification received by the terminal.

[0977] (Application example 1)

[0978] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0979] Conventional systems lack the ability to quickly analyze a user's health status and recommend appropriate medical institutions and health-related products. As a result, users have no means to respond immediately to their health status and spend a lot of time and effort finding appropriate medical services and health products. The present invention aims to solve this problem by providing a system that allows users to quickly recognize health abnormalities and receive recommendations for appropriate medical institutions and health-related products.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0981] In this invention, the server includes means for collecting voice data and search history data from the user's computer terminal, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting words or phrases indicating health abnormalities, means for identifying an appropriate medical facility based on the words or phrases and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's computer terminal, means for coordinating with the medical facility's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's computer terminal of details of the appointment, and means for analyzing the user's health condition data and suggesting health-related products. This allows the user to quickly respond to their health condition and receive appointments at appropriate medical institutions and suggestions for health-related products.

[0982] A "user" is a person who uses this system and provides voice data and search history data.

[0983] "Computer terminal" means a device used by a user, and is a digital device including a smartphone, tablet, PC, smart glasses, etc.

[0984] "Voice data" refers to voice information uttered by a user into a computer terminal and analyzed using voice recognition technology.

[0985] "Search history data" refers to the history of searches a user has conducted on the Internet, and is used to infer the user's interests and health status.

[0986] "Character data" is information obtained by analyzing voice data and converting it into text.

[0987] "Terms" are keywords and phrases extracted from voice data and search history data, and are used to identify health abnormalities and suggest products.

[0988] A "medical facility" is a facility where a person can receive medical treatment from a doctor, and refers to general forms of medical institutions, including hospitals and clinics.

[0989] The "available appointment dates and times" are specific dates and times when a medical facility can accept a user's appointment.

[0990] "Reservation proposal" refers to the content of a proposal for a medical facility reservation that the server sends to the user based on the analysis results.

[0991] "Health Status Data" refers to information related to a user's health that is inferred based on goad cues obtained from voice data and search history data.

[0992] "Health-related products" are products suggested based on the user's health condition, including supplements, medicines, health foods, etc.

[0993] Overall overview

[0994] As an embodiment of the present invention, the system program is designed as follows.

[0995] System Configuration and Operation

[0996] Data collection

[0997] First, the user inputs their health condition into a computer terminal by voice. For example, the user might say, "I've been feeling tired lately." This voice data is converted into text data by voice recognition software (e.g., Google Cloud Speech-to-Text API) running on the computer terminal. At the same time, if the user searches online for something like "ways to recover from fatigue," that search history data is also collected.

[0998] Data analysis

[0999] The server receives the converted text data and search history data and analyzes it using natural language processing (NLP) technology (e.g., Transformers with the BERT model). This analysis extracts words that indicate health abnormalities. For example, the analysis results may include words such as "fatigue" and "rest."

[1000] Identifying and recommending medical facilities and products

[1001] Based on the analysis results, the server takes into account the user's location information and identifies appropriate medical facilities and health-related products, such as nearby internal medicine clinics or supplements effective for fatigue recovery. This information is then sent to the computer terminal along with available appointment times and detailed product information.

[1002] User Notification and Response

[1003] The user receives information about the proposed medical facility and product on the computer terminal. For example, a message such as "Would you like to purchase a fatigue recovery supplement?" or "I have made an appointment with a nearby internal medicine clinic. Would you like to confirm the appointment?" is displayed. If the user accepts the appointment suggestion, the server connects with the medical facility's reservation system to confirm the appointment. If the user responds to the product suggestion, the purchase procedure is carried out.

[1004] Reservation details and reminders

[1005] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. For example, it may say, "Your reservation has been made at the internal medicine clinic on a certain date at a certain time." In addition, when the reservation date and time approaches, a reminder notification is generated and sent again to the computer terminal to notify the user.

[1006] Specific examples

[1007] A user says to their smartphone, "I have a headache."

[1008] 1. Speech recognition software converts the speech into the text "I have a headache."

[1009] 2. The converted character data is sent to the server.

[1010] 3. The server analyzes the text data and search history data to extract words that indicate health abnormalities, such as "headache."

[1011] 4. Based on the analysis results, medical facilities and health products related to headaches are identified, and detailed information is sent from the server to the smartphone.

[1012] 5. The user responds to prompts such as "Would you like to make an appointment with a local doctor?" or "Would you like to buy some headache medicine?"

[1013] 6. Once the user approves the reservation, the server connects to the medical facility's reservation system to confirm the reservation.

[1014] 7. The server will notify your smartphone of the reservation details.

[1015] 8. When your appointment time approaches, a reminder notification will be sent to your smartphone.

[1016] Prompt Sentence Examples

[1017] "Please generate an application program that analyzes the user's health-related voice data and suggests appropriate medical institutions and health products. For example, if a user says, 'I have a headache,' convert the voice into text, analyze that data, and suggest related products."

[1018] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1019] Step 1:

[1020] A user inputs voice into a computer terminal. At this time, the computer terminal uses voice recognition software (e.g., Google Cloud Speech-to-Text API) to collect the voice data and convert it into text data. The input is the user's voice data, and the output is the converted text data.

[1021] Step 2:

[1022] The terminal sends the converted text data to the server. At the same time, the user's recent search history data is also sent to the server. The input is the text data and the search history data, and the output is the data sent to the server.

[1023] Step 3:

[1024] The server analyzes the received text data and search history data using natural language processing (NLP) techniques (e.g., Transformers with the BERT model). This analysis extracts phrases that indicate health abnormalities. The input is the text data and search history data, and the output is the extracted health abnormality phrases.

[1025] Step 4:

[1026] The server identifies appropriate medical facilities based on the analysis results and takes into account the user's location information. At the same time, it searches the database for health-related products related to the health abnormality. The input is the health abnormality phrase and location information, and the output is a list of identified medical facilities and health-related products.

[1027] Step 5:

[1028] The server creates reservation proposals and product proposals, including available appointment dates and times and detailed product information, and notifies the user of these proposals. The input is detailed information about medical facilities and products, and the output is a notification to the user.

[1029] Step 6:

[1030] The user checks the proposed content on the computer terminal and responds, for example, by answering "yes" or "no" to messages such as "Would you like to make a reservation?" or "Would you like to purchase a product?" The input at this step is the user response, and the output is the user's choice.

[1031] Step 7:

[1032] If the user accepts the reservation proposal, the server will confirm the reservation in cooperation with the medical facility's reservation system. If the user accepts the product proposal, the server will proceed with the purchase procedure. The input is the user's selection, and the output is the confirmed reservation and a notification of purchase completion.

[1033] Step 8:

[1034] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. Furthermore, when the reservation date and time approaches, a reminder notification is generated and sent to the user's computer terminal. The input is the confirmed reservation information, and the output is the reservation details notification and the reminder notification.

[1035] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1036] In one embodiment of the present invention, we design an automated system that collects voice data and search history data from a user's smart device, analyzes this data to detect health abnormalities, recommends appropriate medical institutions, and schedules appointments. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to perform additional analysis and judgment based on the user's emotional state.

[1037] Overall overview

[1038] This system consists of a terminal, a server, and a user. The terminal is a smart device that collects voice data and search history data. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. An emotion engine has also been added to evaluate the user's emotional state and complement the health abnormality analysis results.

[1039] System Configuration and Operation

[1040] Data collection

[1041] The device collects voice data and search history data every time the user speaks to the smart device or searches the internet. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device will record the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device will also collect that search history data.

[1042] Data Conversion and Transmission

[1043] The collected voice data is converted into text data using the device's voice recognition function, and the converted text data and search history data are sent to the server.

[1044] Data analysis

[1045] The server analyzes the text data and search history data sent to it. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[1046] Emotion recognition and evaluation

[1047] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine evaluates the user's emotional state based on their vocabulary, tone of voice, search results, etc. For example, if a user says, "I'm having a really hard time sleeping," the engine detects their feelings of confusion and anxiety.

[1048] Identifying medical institutions and suggesting appointments

[1049] Based on the analysis results and emotion evaluation results, the server searches for suitable nearby medical institutions, taking into account the user's location information. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[1050] User Notification

[1051] The device displays a notification of the appointment suggestion sent from the server to the user. For example, a message such as "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?" can be displayed. The user can respond to this message with "Yes" or "No."

[1052] Confirmation of reservation and notification of details

[1053] If the user answers "yes," the server will connect to the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) will be notified to the terminal.

[1054] Reminder function

[1055] When the appointment time approaches, the server generates a reminder notification and sends it to the device, which displays the reminder notification to the user to remind them not to forget to come to the appointment.

[1056] Specific examples

[1057] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[1058] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I haven't been sleeping well lately," to the server.

[1059] 3. At the same time, the user's browser search history for "insomnia relief" is also sent to the server.

[1060] 4. The server analyzes keywords such as "can't sleep well" and "insomnia" and also evaluates the user's emotional state.

[1061] 5. Based on the analysis results, search for nearby sleep clinics and obtain available appointment dates and times.

[1062] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[1063] 7. If the user responds "yes," the server confirms the appointment with the designated sleep clinic.

[1064] 8. The terminal displays detailed information such as "An appointment has been made for the sleep clinic at XX / XX / XX at XX time."

[1065] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[1066] The present invention, which combines emotion engines in this way, enables quick responses to the user's health condition, selection of an appropriate medical institution, and support for consultation.

[1067] The processing flow will be explained below.

[1068] Step 1:

[1069] The user speaks a health-related phrase into their smart device (e.g., "I haven't been sleeping well lately").

[1070] Step 2:

[1071] The device uses voice recognition to capture the user's voice data and convert it into text data.

[1072] Step 3:

[1073] The device retrieves the user's search history data from the browser.

[1074] Step 4:

[1075] The terminal transmits the converted text data and the acquired search history data to the server.

[1076] Step 5:

[1077] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[1078] Step 6:

[1079] The server uses an emotion engine to recognize the user's emotion from the voice data and text data and evaluates the user's emotional state.

[1080] Step 7:

[1081] The server evaluates the user's health abnormalities and determines the risk based on the extracted keywords and sentiment analysis results.

[1082] Step 8:

[1083] The server references the user's location information and searches for appropriate medical facilities nearby.

[1084] Step 9:

[1085] The server obtains the available appointment dates and times for the medical institutions found by the search.

[1086] Step 10:

[1087] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[1088] Step 11:

[1089] The device displays a notification message from the server to the user (e.g., "You haven't been sleeping well lately. Would you like to make an appointment with a nearby sleep clinic?").

[1090] Step 12:

[1091] The user responds to the notification with a "yes" or "no" answer.

[1092] Step 13:

[1093] The terminal sends the user's response results to the server.

[1094] Step 14:

[1095] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[1096] Step 15:

[1097] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[1098] Step 16:

[1099] The terminal displays a reservation confirmation message from the server to the user.

[1100] Step 17:

[1101] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[1102] Step 18:

[1103] The device displays a reminder notification from the server to the user.

[1104] Specifically, when a user says, "I haven't been sleeping well lately," the device converts the voice data into text and sends it to the server. The server analyzes the text data, extracts keywords such as "can't sleep" and "insomnia," and uses an emotion engine to evaluate the user's feelings of anxiety and confusion. Based on this, the device searches for appropriate medical institutions and their available appointment dates and times, and makes a reservation suggestion. If the user responds "yes," the reservation is confirmed. The reservation details and a reminder notification are then sent to the user.

[1105] Example 2

[1106] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1107] In modern society, it is extremely important for users to take prompt and appropriate measures regarding their health. However, the process of users recognizing their own health abnormalities and finding an appropriate medical institution is complicated and time-consuming. Furthermore, a user's emotional state and stress level also have a significant impact on their health, but there is a lack of responses that take these into consideration. Therefore, there is a need for a system that efficiently utilizes a user's voice data and search history data to detect health abnormalities, identify appropriate medical institutions, and assist in making appointments, while also taking their emotional state into consideration.

[1108] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting voice data and digital internet information from the user's information processing device; means for converting the voice data into text data; means for analyzing the text data and digital internet information and extracting an identifier indicating a health abnormality; means for identifying an appropriate medical facility and obtaining an available appointment date and time based on the identifier; means for sending a notification including an appointment proposal to the user's information processing device; means for confirming the appointment in cooperation with the medical facility's appointment system if the user accepts the appointment proposal; means for notifying the user's information processing device of details of the appointment; and means including an emotion engine that recognizes the user's emotional state and performs complementary analysis based on the analysis results. This allows the user to respond to their health condition quickly and appropriately, and accurate detection of health abnormalities and recommendations for medical institutions are realized taking the user's emotional state into consideration.

[1109] "User's information processing device" refers to an electronic device used by a user, such as a smartphone, tablet, or personal computer, that has the function of collecting and transmitting voice data and digital internet information.

[1110] "Voice data" refers to data that is recorded in digital format via a microphone of what a user says to an information processing device.

[1111] "Digital Internet information" refers to historical data such as the search history a user has conducted on the Internet and the web pages they have accessed.

[1112] The "means for converting into text data" refers to a speech recognition technology or system for converting voice data into text format.

[1113] "Identifiers" are keywords or phrases extracted from the data being analyzed that are used to indicate specific health abnormalities.

[1114] "Medical facilities" are medical institutions such as hospitals and clinics that provide medical treatment according to the user's health condition.

[1115] An "emotion engine" is a technology or system that recognizes a user's emotional state based on their voice data and text data, and provides analysis results according to their emotions.

[1116] A "reservation system" is a system used by medical facilities to manage reservations, and has functions for accepting, managing, and confirming reservations.

[1117] A "notification" is a message sent from the server to the user's information processing device, and includes information such as a medical facility reservation suggestion, detailed reservation information, and a reminder.

[1118] "Emotional state" refers to the psychological state analyzed from the user's voice and text data, and indicates emotions such as joy, sadness, anger, and anxiety.

[1119] "Complementary analytics" refers to additional or detailed analytics based on a user's emotional state that can be used to make more accurate decisions or recommendations.

[1120] As an embodiment of the present invention, we have designed an automated system that collects voice data and digital internet information from a user's information processing device, analyzes this data to detect health abnormalities, recommends appropriate medical facilities, and makes appointments. This system also incorporates an emotion engine that recognizes the user's emotional state, allowing it to perform additional analysis and judgment based on the user's emotional state.

[1121] Overall overview

[1122] This system consists of the user's information processing device (smart device), a server, and the user's components. The user's information processing device collects voice data and digital internet information. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. In addition, an emotion engine has been added to evaluate the user's emotional state and complement the analysis results of health abnormalities.

[1123] Hardware and software used

[1124] Information processing equipment: smart devices (smartphones, tablets, etc.)

[1125] Server: Dedicated server for data analysis

[1126] Speech recognition technology: Google Speech-to-Text API, etc.

[1127] NLP engines: spaCy and NLTK built in Python

[1128] Emotion recognition engine: Uses Praat and openSMILE as voice analysis tools

[1129] Geographic information service: Google Maps API

[1130] Push notification service: Firebase Cloud Messaging

[1131] Natural language description of the process

[1132] The device collects voice data when the user speaks to the smart device and the search history data they perform on the Internet. For example, if a user says, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device collects the search history.

[1133] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are then sent to a server. The server then uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[1134] The server uses an emotion engine to recognize the user's emotional state from text and voice data. For example, if someone says, "I'm having a really hard time sleeping," the server can detect feelings of confusion and anxiety. This emotion assessment is used to complement the analysis results.

[1135] Based on the analysis results and emotion evaluation results, the server takes into account the user's location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates an appointment suggestion. The device displays the appointment suggestion notification sent from the server to the user. For example, a message may appear saying, "You seem to be having trouble sleeping. Would you like to make an appointment at a nearby sleep clinic?" The user can respond to this message with "Yes" or "No."

[1136] If the user responds "Yes," the server will link with the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, detailed information (date, time, location, name of medical institution, etc.) will be notified to the device. When the appointment date and time approaches, the server will generate a reminder notification and send it to the device. The device will display this reminder notification to the user to remind them not to forget to visit.

[1137] Specific examples

[1138] Example 1:

[1139] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[1140] 2. The device converts the speech into text using a speech recognition function (for example, Google Speech-to-Text API) and sends the text data, such as "I haven't been sleeping well lately," to the server.

[1141] 3. At the same time, the user's browser search history data for "insomnia relief" is sent to the server.

[1142] 4. The server uses a Python NLP engine (e.g., spaCy) to analyze keywords such as "can't sleep well" and "insomnia" and also evaluate the user's emotional state.

[1143] 5. Based on the analysis results, use the Google Maps API to search for nearby sleep clinics and obtain available appointment dates and times.

[1144] 6. Using Firebase Cloud Messaging, display a notification on the device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[1145] 7. If the user answers "Yes," the server calls the medical institution's reservation API to confirm the appointment with the specified sleep clinic.

[1146] 8. The device will display detailed information such as "Your appointment has been made at the sleep clinic on XX / XX / XX at XX time." A reminder notification will also be scheduled.

[1147] Prompt Sentence Examples

[1148] Please explain the process of suggesting an appointment for a sleep clinic to a user who says, "I haven't been sleeping well lately."

[1149] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1150] Step 1:

[1151] The device collects voice data when the user speaks to the smart device and the search history data of the user's internet searches. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "insomnia relief," the device collects the search history.

[1152] Input: User voice data, internet search history data

[1153] Output: Recorded voice data, collected search history data

[1154] Step 2:

[1155] The collected voice data is converted into text data using the device's voice recognition function. For example, the Google Speech-to-Text API is used to convert the speech "I haven't been sleeping well lately" into text. The converted text data and search history data are then sent to the server.

[1156] Input: Recorded audio data

[1157] Output: Text data converted to character data, search history data

[1158] Step 3:

[1159] The server analyzes the text data and search history data sent to it. It uses natural language processing (NLP) technology, such as a Python NLP engine (spaCy or NLTK), to extract keywords that indicate health abnormalities. For example, it could extract keywords such as "not sleeping well" or "insomnia" and determine that the user has sleep problems.

[1160] Input: Text data converted to character data, search history data

[1161] Output: Extracted keywords related to health abnormalities

[1162] Step 4:

[1163] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine uses speech analysis tools (Praat and openSMILE) to analyze the intonation, speed, and volume of the voice data and extract emotions from the text data. For example, if a user says, "I'm having a really hard time sleeping," the emotion engine will detect confusion and anxiety.

[1164] Input: Text data converted to character data, audio data

[1165] Output: Data about the user's emotional state

[1166] Step 5:

[1167] Based on the analysis results and emotion evaluation results, the server searches for appropriate nearby medical institutions, taking into account the user's location information. For example, it uses the Google Maps API to search for nearby sleep clinics based on the user's location information and obtains available appointment dates and times at each medical institution.

[1168] Input: location information, data analysis results, emotion evaluation results

[1169] Output: List of suitable medical institutions, available appointment dates and times

[1170] Step 6:

[1171] The device displays the suggestions for available medical institutions sent from the server as a notification to the user. For example, it uses Firebase Cloud Messaging to notify the user with a message such as, "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?"

[1172] Input: List of suitable medical institutions, available appointment dates and times

[1173] Output: Notification of booking proposal to user

[1174] Step 7:

[1175] If the user accepts the reservation proposal, the server cooperates with the medical institution's reservation system to confirm the reservation at the specified date and time, for example, through the medical institution's reservation API.

[1176] Input: User response

[1177] Output: Confirmed reservation information

[1178] Step 8:

[1179] After the reservation is successfully made, the details (date, time, location, name of medical institution, etc.) are notified to the device. In addition, when the reservation date and time approaches, the server generates a reminder notification and sends it to the device. The device displays this reminder notification to the user.

[1180] Input: Confirmed reservation information, schedule information for reminder notifications

[1181] Output: Reservation details notification, reminder notification

[1182] (Application example 2)

[1183] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] Conventional health management systems and food delivery services have been unable to effectively utilize users' voice data and search history data to detect health abnormalities or recommend appropriate medical institutions. Furthermore, they have not made meal suggestions that take into account the user's emotional state, making it difficult for users to select meals based on their own health condition and emotions. This has led to the issue of users being unable to receive appropriate support to maintain and improve their health.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1186] In this invention, the server includes means for collecting voice data and search history data from the user's information processing device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicating health abnormalities, means for identifying appropriate medical institutions based on the keywords and obtaining available appointment dates and times, means for suggesting appropriate meals based on the user's health abnormalities and emotional state, and means for transmitting the contents of the suggested meals to the information processing device. This allows the user to receive meal suggestions based on their health and emotional state, and also enables them to quickly make appointments at necessary medical institutions.

[1187] An "information processing device" is an electronic device used by a user, such as a smartphone, tablet, or computer, and is a device for collecting and analyzing voice data and search history data.

[1188] "Voice data" refers to digital data that records what a user says to an information processing device.

[1189] "Search history data" refers to data that records the history of a user's internet searches.

[1190] "Text data" refers to data obtained by converting voice data into character information.

[1191] "Keywords" are important words that indicate health abnormalities or emotional states, extracted from text data and search history data.

[1192] A "medical institution" is an institution that provides medical services, such as a hospital or clinic.

[1193] An "emotion engine" is software that analyzes a user's emotional state from their voice and text data.

[1194] "Meal suggestions" are information that recommends optimal meals based on the user's health and emotional state.

[1195] A "notification" is a message or an alert sent to an information processing device.

[1196] A "reservation system" is a system for managing and processing reservations at medical institutions.

[1197] As an embodiment of the present invention, a system applied to a food delivery service consisting of a user, a terminal, and a server will be described. The operation of each element and the overall flow will be described in detail below.

[1198] Configuration and Operation

[1199] Data collection

[1200] The device collects voice data entered by the user using the voice recognition function. For example, if the user says, "I've been feeling tired lately," the device records the voice and saves it as voice data. At the same time, it collects the keywords the user searched for (e.g., "how to relieve fatigue") as search history data.

[1201] Data Conversion and Transmission

[1202] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are sent to the server. For voice recognition, the speech_recognition library, for example, is used.

[1203] Data analysis

[1204] The server uses natural language processing (NLP) technology to analyze the submitted text data and search history data, extracting keywords that indicate health abnormalities (e.g., "fatigue" and "easily tired"). The analysis uses a health API (e.g., https: / / myhealthapi.example.com).

[1205] Emotion recognition and evaluation

[1206] The server identifies the user's emotional state using an emotion engine. This emotion engine evaluates the user's emotion from text data and voice tone. For emotion analysis, an emotion analysis API (e.g., https: / / myfeelingsapi.example.com) is used.

[1207] Meal suggestions

[1208] Based on the analyzed health and emotion data, the server suggests appropriate meals using a meal suggestion API (e.g., https: / / mydeliveryapi.example.com).

[1209] User Notification

[1210] The device will notify the user of the meal suggestions sent from the server, for example, by displaying a message saying, "We will suggest nutritious meals that suit your condition."

[1211] Specific examples

[1212] For example, if a user says, "I've been feeling a bit tired lately," the system analyzes the voice data and determines that the user is tired. Next, it performs emotion analysis to evaluate the user's emotions. Based on this, it suggests nutritious meals (e.g., vitamin-rich meals) and notifies the information processing device so that the user can immediately order the meals.

[1213] Prompt Sentence Examples

[1214] "User voice data: 'I've been feeling a bit tired lately.' Search history: 'How to relieve fatigue.' Analyze this data to suggest nutritious meals that are suitable for the user."

[1215] In this way, users can receive optimal support based on their health and emotional state.

[1216] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1217] Step 1:

[1218] The device collects voice data entered by the user using the voice recognition function. The input requires the user's voice, such as saying, "I've been feeling tired lately." The voice data is stored as digital data on the device. In addition, if the user enters search keywords such as "how to relieve fatigue," the device also collects search history data.

[1219] Step 2:

[1220] The device converts the collected voice data into text data using a voice recognition function. Digital voice data is used as input, and the conversion is performed using, for example, the speech_recognition library. The output is string data that can be analyzed from the voice data. At the same time, the converted text data and search history data are sent to the server.

[1221] Step 3:

[1222] The server analyzes the submitted text data and search history data using natural language processing (NLP) techniques. The input is the text data and search history data, and keywords (e.g., "fatigue" and "easily tired") are extracted using a health API (e.g., https: / / myhealthapi.example.com). The extracted keywords are obtained as output.

[1223] Step 4:

[1224] The server uses an emotion engine to analyze the user's emotions from the text data. It uses the text data obtained in step 3 as input and utilizes an emotion analysis API (e.g., https: / / myfeelingsapi.example.com). It outputs the user's emotional state (e.g., "stressed" or "tired").

[1225] Step 5:

[1226] The server suggests appropriate meals based on the health and emotional state data. As input, it uses keywords indicating health abnormalities and the emotional state data and sends a request to the meal suggestion API (e.g., https: / / mydeliveryapi.example.com). The output is a suggested meal menu.

[1227] Step 6:

[1228] The device notifies the user of the meal suggestions sent from the server. As input, it receives the suggestion data from the server and displays a message to the user saying, "We will suggest nutritious meals that suit your condition." The output is a notification message that the user can see.

[1229] This allows users to receive appropriate health support.

[1230] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1231] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1232] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1233] [Fourth embodiment]

[1234] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1235] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1236] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1237] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1238] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1239] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1240] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1241] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1242] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1243] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1244] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1245] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1246] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1247] As an embodiment of the present invention, the system program is designed as follows.

[1248] Overall overview

[1249] This automated system collects voice data and search history data from users' smart devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of terminals, servers, and users.

[1250] System Configuration and Operation

[1251] Data collection

[1252] The device collects voice input and search history every time a user speaks to or searches on the smart device. For example, if a user says "my tooth hurts" to their smartphone, the device records the voice data. At the same time, if the user searches for keywords such as "toothache causes" on the browser, the device also collects that history data.

[1253] Data Conversion and Transmission

[1254] The collected voice data is converted into text data using the device's voice recognition function, and this converted text data and search history data are then sent to the server.

[1255] Data analysis

[1256] The server analyzes the submitted text data and search history data. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "toothache" or "tooth decay" are recognized, the server will determine that this is a type of health abnormality.

[1257] Identifying medical institutions and suggesting appointments

[1258] Based on the analysis results, the server takes into account the user's current location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[1259] User Notification

[1260] The device displays the appointment suggestion notification sent from the server to the user. For example, a message such as "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?" can be displayed. The user can choose to respond with "Yes" or "No" to this message.

[1261] Confirmation of reservation and notification of details

[1262] If the user answers "yes," the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) are notified to the terminal.

[1263] Reminder function

[1264] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[1265] Specific examples

[1266] 1. The user says to their smartphone, "I've been having stomach aches lately."

[1267] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[1268] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[1269] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[1270] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[1271] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[1272] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[1273] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[1274] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[1275] Through this series of processes, users can quickly recognize any health abnormalities and ensure that they are referred to an appropriate medical institution.

[1276] The processing flow will be explained below.

[1277] Step 1:

[1278] The user speaks a health-related phrase (e.g., "My tooth hurts") into their smart device.

[1279] Step 2:

[1280] The device uses voice recognition to capture the user's voice data and convert it into text data.

[1281] Step 3:

[1282] The device retrieves the user's search history data from the browser.

[1283] Step 4:

[1284] The terminal transmits the converted text data and the acquired search history data to the server.

[1285] Step 5:

[1286] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[1287] Step 6:

[1288] The server searches for an appropriate medical institution using the user's location information based on the extracted keywords.

[1289] Step 7:

[1290] The server obtains the available appointment dates and times for the medical institutions found by the search.

[1291] Step 8:

[1292] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[1293] Step 9:

[1294] The device displays a notification message from the server to the user (e.g., "You seem to have a toothache. Would you like to make an appointment with a nearby dentist?").

[1295] Step 10:

[1296] The user responds to the notification with a "yes" or "no" answer.

[1297] Step 11:

[1298] The terminal sends the user's response results to the server.

[1299] Step 12:

[1300] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[1301] Step 13:

[1302] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[1303] Step 14:

[1304] The terminal displays a reservation confirmation message from the server to the user.

[1305] Step 15:

[1306] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[1307] Step 16:

[1308] The device displays a reminder notification from the server to the user.

[1309] Example 1

[1310] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1311] In modern society, there are an increasing number of situations where users need to quickly understand their own health status and seek medical attention at an appropriate medical institution. However, users often need a great deal of time and effort to collect medical information, and there is a lack of appropriate systems for selecting and booking medical institutions. In particular, the time required for early detection of health abnormalities and selecting an appropriate medical institution is an issue.

[1312] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1313] In this invention, the server includes means for collecting voice data and search history data from the user's electronic device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicative of health abnormalities, means for identifying an appropriate medical institution based on the keywords and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's electronic device, means for cooperating with the medical institution's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's electronic device of detailed information about the appointment, and means for generating and sending a reminder notification when the appointment date and time approaches. This allows the user to quickly understand their own health abnormalities and ensure that they visit an appropriate medical institution.

[1314] "User" refers to an individual or organization that uses an electronic device.

[1315] "Electronic terminal" refers to devices with communication capabilities, such as smart devices, personal computers, and tablets.

[1316] "Audio data" refers to digital data acquired as audio input.

[1317] "Search History Data" refers to data that includes the history of searches you perform on your internet browser or applications.

[1318] "Text data" refers to data in which voice data has been converted into a string of characters.

[1319] "Health abnormalities" refers to any abnormal or unwell health condition experienced by the User.

[1320] "Keywords" refer to specific words or phrases related to a health condition.

[1321] "Medical institution" refers to a facility that provides medical services, such as a doctor's office, hospital, or clinic.

[1322] "Available appointment dates" refers to dates and times when a medical institution can accept appointments for medical treatment.

[1323] "Notification" means the sending of a message or alert to a User's electronic device.

[1324] "Reminder notification" refers to notifying the user again when the reservation deadline approaches.

[1325] "Server" refers to a computer system that stores, manages, analyzes, and communicates data over a network.

[1326] As an embodiment of the present invention, the system program is implemented as follows.

[1327] System Overview

[1328] This automated system collects voice data and search history data from users' electronic devices, analyzes this data to detect health abnormalities, and suggests appropriate medical institutions and makes appointments.The system is primarily composed of three elements: the device, the server, and the user.

[1329] Data collection and transformation

[1330] The device collects voice input and search history every time a user speaks to the device or searches. For example, if a user says to the device, "I've had a stomach ache recently," the device records the voice data. Similarly, if a user searches for "causes of stomach ache" on a browser, the device also collects that history data.

[1331] The collected voice data is converted into text data using the device's voice recognition function (e.g., a cloud-based voice recognition API). Specifically, the Google Cloud Speech-to-Text API can be used. This converted text data and search history data are then sent to a server.

[1332] Data analysis

[1333] The server analyzes the text data and search history data sent to it. Natural language processing (NLP) technology is used for the analysis. For example, keywords indicating health abnormalities are extracted using the Google Cloud Natural Language API. Specifically, keywords such as "my stomach hurts" and "causes of stomach pain" are extracted from the text data and search history data, and the presence or absence of health abnormalities is determined based on this.

[1334] Identifying medical institutions and suggesting appointments

[1335] Based on the analysis results, the server takes into account the user's current location information (e.g., GPS data) and searches for suitable nearby medical institutions. It then obtains available appointment dates and times for each medical institution and creates an appointment proposal. The appointment proposal includes a specific message such as "Would you like to make an appointment at a nearby internal medicine clinic?" The proposal is then sent to the device as a notification.

[1336] User notification and booking confirmation

[1337] The terminal displays the appointment suggestion notification sent from the server to the user. If the user responds "Yes" to this message, the server connects to the medical institution's reservation system and confirms the appointment for the specified date and time. After the appointment is successfully made, the terminal is notified of the details (date, time, location, name of the medical institution, etc.).

[1338] Reminder function

[1339] When the appointment time approaches, the server generates a reminder notification and sends it to the device again. The device displays this reminder notification to the user to remind them not to forget to come to the appointment.

[1340] Specific examples

[1341] 1. The user speaks to an electronic device saying, "I've been having stomach aches lately."

[1342] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I've been having stomach aches lately," to the server.

[1343] 3. At the same time, the user's browser search history for "causes of stomach pain" is also sent to the server.

[1344] 4. The server analyzes keywords such as "stomach ache" and "causes of stomach ache" to detect any health abnormalities in the user.

[1345] 5. Based on the analysis results, search for nearby internal medicine medical institutions and obtain available appointment dates and times.

[1346] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby internal medicine clinic?"

[1347] 7. If the user responds "yes," the server confirms the appointment with the designated internal medicine clinic.

[1348] 8. A notification will appear on the device saying, "An appointment has been made at the internal medicine clinic on XX / XX / XX at XX time."

[1349] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[1350] Prompt Sentence Examples

[1351] Example prompt sentence:

[1352] Please describe a system that detects health abnormalities and schedules appropriate medical appointments by having users talk to their smartphones or look at their search history. Please include the following information: the hardware and software used, what data is collected and analyzed, how it is collected and analyzed, how users are notified, and specific examples.

[1353] Through the above-described procedures, the present invention assists the user in managing his or her health and ensures prompt and appropriate medical treatment.

[1354] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1355] Step 1: Data collection

[1356] A user speaks to an electronic device or performs a search. The user's voice data and search history data are obtained as input. The device records the voice data when the user speaks to the electronic device, such as "I've had a stomachache recently," and collects search history data for "causes of stomach pain" on the browser. The recorded voice data and search history data are generated as output.

[1357] Step 2: Data conversion

[1358] The device converts the collected voice data into text data using a speech recognition function (for example, Google Cloud Speech-to-Text API). The recorded voice data is given as input. As a result of the conversion, the text data "I've had a stomach ache recently" is output. Furthermore, the search history data is used in its original form.

[1359] Step 3: Sending data

[1360] The terminal transmits the converted text data and search history data to the server. As input, the text data and search history data are provided. These data are encrypted and transmitted to the server. As output, the text data and search history data received by the server are obtained.

[1361] Step 4: Data analysis

[1362] The server analyzes the received text data and search history data to extract keywords that indicate health abnormalities. As input, the received text data "I've had a stomach ache recently" and search history data "Causes of stomach pain" are given. The server uses the Google Cloud Natural Language API to extract keywords such as "my stomach hurts" and "stomach pain" from these data. The extracted keywords are generated as output.

[1363] Step 5: Identify medical facilities

[1364] The server determines health abnormalities based on the extracted keywords and identifies appropriate medical institutions. The extracted keywords and the user's current location information (e.g., GPS data) are given as input. The server searches for nearby medical institutions and obtains available appointment dates and times. The output is the medical institution and available appointment date and time information.

[1365] Step 6: Creating a booking proposal

[1366] The server creates an appointment suggestion based on the retrieved medical institution and available appointment date and time. Medical institution information and available appointment date and time information are given as input. The server creates an appointment suggestion such as "Would you like to make an appointment at a nearby internal medicine clinic?". As output, an appointment suggestion notification is generated.

[1367] Step 7: Send booking proposal notification

[1368] The server sends the created reservation proposal notification to the terminal. The reservation proposal notification is given as input. The server sends the reservation proposal notification to the user's terminal. The reservation proposal notification received by the terminal is obtained as output.

[1369] Step 8: Receiving user response

[1370] The user responds to the reservation suggestion notification displayed on the terminal. As input, the user's response "yes" or "no" is obtained. For example, the user responds "yes." As output, the user's response is generated.

[1371] Step 9: Confirm your booking

[1372] The server receives the user's response and confirms the appointment by linking with the medical institution's appointment system. The user's response and the medical institution's appointment information are given as input. The server confirms the appointment for the specified date and time through the medical institution's online appointment system. The detailed appointment information is generated as output.

[1373] Step 10: Send reservation details notification

[1374] The server sends the details of the confirmed reservation to the terminal. As input, the reservation details are given. The server sends the reservation details to the user's terminal. As output, a reservation details notification received by the terminal is obtained.

[1375] Step 11: Generate a reminder notification

[1376] The server generates a reminder notification when the reservation date and time is approaching. Reservation date and time information is given as input. The server generates the reminder notification. The generated reminder notification is obtained as output.

[1377] Step 12: Send reminders

[1378] The server sends the generated remind notification to the terminal. The remind notification is given as input. The server sends the remind notification to the user's terminal. The output is the remind notification received by the terminal.

[1379] (Application example 1)

[1380] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1381] Conventional systems lack the ability to quickly analyze a user's health status and recommend appropriate medical institutions and health-related products. As a result, users have no means to respond immediately to their health status and spend a lot of time and effort finding appropriate medical services and health products. The present invention aims to solve this problem by providing a system that allows users to quickly recognize health abnormalities and receive recommendations for appropriate medical institutions and health-related products.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1383] In this invention, the server includes means for collecting voice data and search history data from the user's computer terminal, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting words or phrases indicating health abnormalities, means for identifying an appropriate medical facility based on the words or phrases and obtaining available appointment dates and times, means for sending a notification including an appointment proposal to the user's computer terminal, means for coordinating with the medical facility's appointment system to confirm the appointment if the user accepts the appointment proposal, means for notifying the user's computer terminal of details of the appointment, and means for analyzing the user's health condition data and suggesting health-related products. This allows the user to quickly respond to their health condition and receive appointments at appropriate medical institutions and suggestions for health-related products.

[1384] A "user" is a person who uses this system and provides voice data and search history data.

[1385] "Computer terminal" means a device used by a user, and is a digital device including a smartphone, tablet, PC, smart glasses, etc.

[1386] "Voice data" refers to voice information uttered by a user into a computer terminal and analyzed using voice recognition technology.

[1387] "Search history data" refers to the history of searches a user has conducted on the Internet, and is used to infer the user's interests and health status.

[1388] "Character data" is information obtained by analyzing voice data and converting it into text.

[1389] "Terms" are keywords and phrases extracted from voice data and search history data, and are used to identify health abnormalities and suggest products.

[1390] A "medical facility" is a facility where a person can receive medical treatment from a doctor, and refers to general forms of medical institutions, including hospitals and clinics.

[1391] The "available appointment dates and times" are specific dates and times when a medical facility can accept a user's appointment.

[1392] "Reservation proposal" refers to the content of a proposal for a medical facility reservation that the server sends to the user based on the analysis results.

[1393] "Health Status Data" refers to information related to a user's health that is inferred based on goad cues obtained from voice data and search history data.

[1394] "Health-related products" are products suggested based on the user's health condition, including supplements, medicines, health foods, etc.

[1395] Overall overview

[1396] As an embodiment of the present invention, the system program is designed as follows.

[1397] System Configuration and Operation

[1398] Data collection

[1399] First, the user inputs their health condition into a computer terminal by voice. For example, the user might say, "I've been feeling tired lately." This voice data is converted into text data by voice recognition software (e.g., Google Cloud Speech-to-Text API) running on the computer terminal. At the same time, if the user searches online for something like "ways to recover from fatigue," that search history data is also collected.

[1400] Data analysis

[1401] The server receives the converted text data and search history data and analyzes it using natural language processing (NLP) technology (e.g., Transformers with the BERT model). This analysis extracts words that indicate health abnormalities. For example, the analysis results may include words such as "fatigue" and "rest."

[1402] Identifying and recommending medical facilities and products

[1403] Based on the analysis results, the server takes into account the user's location information and identifies appropriate medical facilities and health-related products, such as nearby internal medicine clinics or supplements effective for fatigue recovery. This information is then sent to the computer terminal along with available appointment times and detailed product information.

[1404] User Notification and Response

[1405] The user receives information about the proposed medical facility and product on the computer terminal. For example, a message such as "Would you like to purchase a fatigue recovery supplement?" or "I have made an appointment with a nearby internal medicine clinic. Would you like to confirm the appointment?" is displayed. If the user accepts the appointment suggestion, the server connects with the medical facility's reservation system to confirm the appointment. If the user responds to the product suggestion, the purchase procedure is carried out.

[1406] Reservation details and reminders

[1407] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. For example, it may say, "Your reservation has been made at the internal medicine clinic on a certain date at a certain time." In addition, when the reservation date and time approaches, a reminder notification is generated and sent again to the computer terminal to notify the user.

[1408] Specific examples

[1409] A user says to their smartphone, "I have a headache."

[1410] 1. Speech recognition software converts the speech into the text "I have a headache."

[1411] 2. The converted character data is sent to the server.

[1412] 3. The server analyzes the text data and search history data to extract words that indicate health abnormalities, such as "headache."

[1413] 4. Based on the analysis results, medical facilities and health products related to headaches are identified, and detailed information is sent from the server to the smartphone.

[1414] 5. The user responds to prompts such as "Would you like to make an appointment with a local doctor?" or "Would you like to buy some headache medicine?"

[1415] 6. Once the user approves the reservation, the server connects to the medical facility's reservation system to confirm the reservation.

[1416] 7. The server will notify your smartphone of the reservation details.

[1417] 8. When your appointment time approaches, a reminder notification will be sent to your smartphone.

[1418] Prompt Sentence Examples

[1419] "Please generate an application program that analyzes the user's health-related voice data and suggests appropriate medical institutions and health products. For example, if a user says, 'I have a headache,' convert the voice into text, analyze that data, and suggest related products."

[1420] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1421] Step 1:

[1422] A user inputs voice into a computer terminal. At this time, the computer terminal uses voice recognition software (e.g., Google Cloud Speech-to-Text API) to collect the voice data and convert it into text data. The input is the user's voice data, and the output is the converted text data.

[1423] Step 2:

[1424] The terminal sends the converted text data to the server. At the same time, the user's recent search history data is also sent to the server. The input is the text data and the search history data, and the output is the data sent to the server.

[1425] Step 3:

[1426] The server analyzes the received text data and search history data using natural language processing (NLP) techniques (e.g., Transformers with the BERT model). This analysis extracts phrases that indicate health abnormalities. The input is the text data and search history data, and the output is the extracted health abnormality phrases.

[1427] Step 4:

[1428] The server identifies appropriate medical facilities based on the analysis results and takes into account the user's location information. At the same time, it searches the database for health-related products related to the health abnormality. The input is the health abnormality phrase and location information, and the output is a list of identified medical facilities and health-related products.

[1429] Step 5:

[1430] The server creates reservation proposals and product proposals, including available appointment dates and times and detailed product information, and notifies the user of these proposals. The input is detailed information about medical facilities and products, and the output is a notification to the user.

[1431] Step 6:

[1432] The user checks the proposed content on the computer terminal and responds, for example, by answering "yes" or "no" to messages such as "Would you like to make a reservation?" or "Would you like to purchase a product?" The input at this step is the user response, and the output is the user's choice.

[1433] Step 7:

[1434] If the user accepts the reservation proposal, the server will confirm the reservation in cooperation with the medical facility's reservation system. If the user accepts the product proposal, the server will proceed with the purchase procedure. The input is the user's selection, and the output is the confirmed reservation and a notification of purchase completion.

[1435] Step 8:

[1436] After the reservation is confirmed, the server notifies the user's computer terminal of the reservation details. Furthermore, when the reservation date and time approaches, a reminder notification is generated and sent to the user's computer terminal. The input is the confirmed reservation information, and the output is the reservation details notification and the reminder notification.

[1437] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1438] In one embodiment of the present invention, we design an automated system that collects voice data and search history data from a user's smart device, analyzes this data to detect health abnormalities, recommends appropriate medical institutions, and schedules appointments. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to perform additional analysis and judgment based on the user's emotional state.

[1439] Overall overview

[1440] This system consists of a terminal, a server, and a user. The terminal is a smart device that collects voice data and search history data. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. An emotion engine has also been added to evaluate the user's emotional state and complement the health abnormality analysis results.

[1441] System Configuration and Operation

[1442] Data collection

[1443] The device collects voice data and search history data every time the user speaks to the smart device or searches the internet. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device will record the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device will also collect that search history data.

[1444] Data Conversion and Transmission

[1445] The collected voice data is converted into text data using the device's voice recognition function, and the converted text data and search history data are sent to the server.

[1446] Data analysis

[1447] The server analyzes the text data and search history data sent to it. This analysis uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[1448] Emotion recognition and evaluation

[1449] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine evaluates the user's emotional state based on their vocabulary, tone of voice, search results, etc. For example, if a user says, "I'm having a really hard time sleeping," the engine detects their feelings of confusion and anxiety.

[1450] Identifying medical institutions and suggesting appointments

[1451] Based on the analysis results and emotion evaluation results, the server searches for suitable nearby medical institutions, taking into account the user's location information. It then obtains available appointment dates and times from each medical institution and creates a reservation proposal. The proposal is then sent to the device as a notification.

[1452] User Notification

[1453] The device displays a notification of the appointment suggestion sent from the server to the user. For example, a message such as "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?" can be displayed. The user can respond to this message with "Yes" or "No."

[1454] Confirmation of reservation and notification of details

[1455] If the user answers "yes," the server will connect to the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, the details (date, time, location, name of the medical institution, etc.) will be notified to the terminal.

[1456] Reminder function

[1457] When the appointment time approaches, the server generates a reminder notification and sends it to the device, which displays the reminder notification to the user to remind them not to forget to come to the appointment.

[1458] Specific examples

[1459] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[1460] 2. The device uses its voice recognition function to convert the speech into text and sends the text data, such as "I haven't been sleeping well lately," to the server.

[1461] 3. At the same time, the user's browser search history for "insomnia relief" is also sent to the server.

[1462] 4. The server analyzes keywords such as "can't sleep well" and "insomnia" and also evaluates the user's emotional state.

[1463] 5. Based on the analysis results, search for nearby sleep clinics and obtain available appointment dates and times.

[1464] 6. Display a notification on your device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[1465] 7. If the user responds "yes," the server confirms the appointment with the designated sleep clinic.

[1466] 8. The terminal displays detailed information such as "An appointment has been made for the sleep clinic at XX / XX / XX at XX time."

[1467] 9. When the reservation date and time approaches, a reminder notification will be sent to the user.

[1468] The present invention, which combines emotion engines in this way, enables quick responses to the user's health condition, selection of an appropriate medical institution, and support for consultation.

[1469] The processing flow will be explained below.

[1470] Step 1:

[1471] The user speaks a health-related phrase into their smart device (e.g., "I haven't been sleeping well lately").

[1472] Step 2:

[1473] The device uses voice recognition to capture the user's voice data and convert it into text data.

[1474] Step 3:

[1475] The device retrieves the user's search history data from the browser.

[1476] Step 4:

[1477] The terminal transmits the converted text data and the acquired search history data to the server.

[1478] Step 5:

[1479] The server analyzes the received text data and search history data and uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities.

[1480] Step 6:

[1481] The server uses an emotion engine to recognize the user's emotion from the voice data and text data and evaluates the user's emotional state.

[1482] Step 7:

[1483] The server evaluates the user's health abnormalities and determines the risk based on the extracted keywords and sentiment analysis results.

[1484] Step 8:

[1485] The server references the user's location information and searches for appropriate medical facilities nearby.

[1486] Step 9:

[1487] The server obtains the available appointment dates and times for the medical institutions found by the search.

[1488] Step 10:

[1489] The server generates a notification message containing the reservation proposal and sends it to the terminal.

[1490] Step 11:

[1491] The device displays a notification message from the server to the user (e.g., "You haven't been sleeping well lately. Would you like to make an appointment with a nearby sleep clinic?").

[1492] Step 12:

[1493] The user responds to the notification with a "yes" or "no" answer.

[1494] Step 13:

[1495] The terminal sends the user's response results to the server.

[1496] Step 14:

[1497] If the user's response is "yes," the server connects with the reservation system of the corresponding medical institution and confirms the reservation.

[1498] Step 15:

[1499] After the reservation is confirmed, the server generates a confirmation message including detailed information such as the reservation date and time, the name and address of the medical institution, and sends it to the terminal.

[1500] Step 16:

[1501] The terminal displays a reservation confirmation message from the server to the user.

[1502] Step 17:

[1503] When the reservation date and time approaches, the server generates a reminder notification and sends it to the terminal.

[1504] Step 18:

[1505] The device displays a reminder notification from the server to the user.

[1506] Specifically, when a user says, "I haven't been sleeping well lately," the device converts the voice data into text and sends it to the server. The server analyzes the text data, extracts keywords such as "can't sleep" and "insomnia," and uses an emotion engine to evaluate the user's feelings of anxiety and confusion. Based on this, the device searches for appropriate medical institutions and their available appointment dates and times, and makes a reservation suggestion. If the user responds "yes," the reservation is confirmed. The reservation details and a reminder notification are then sent to the user.

[1507] Example 2

[1508] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1509] In modern society, it is extremely important for users to take prompt and appropriate measures regarding their health. However, the process of users recognizing their own health abnormalities and finding an appropriate medical institution is complicated and time-consuming. Furthermore, a user's emotional state and stress level also have a significant impact on their health, but there is a lack of responses that take these into consideration. Therefore, there is a need for a system that efficiently utilizes a user's voice data and search history data to detect health abnormalities, identify appropriate medical institutions, and assist in making appointments, while also taking their emotional state into consideration.

[1510] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting voice data and digital internet information from the user's information processing device; means for converting the voice data into text data; means for analyzing the text data and digital internet information and extracting an identifier indicating a health abnormality; means for identifying an appropriate medical facility and obtaining an available appointment date and time based on the identifier; means for sending a notification including an appointment proposal to the user's information processing device; means for confirming the appointment in cooperation with the medical facility's appointment system if the user accepts the appointment proposal; means for notifying the user's information processing device of details of the appointment; and means including an emotion engine that recognizes the user's emotional state and performs complementary analysis based on the analysis results. This allows the user to respond to their health condition quickly and appropriately, and accurate detection of health abnormalities and recommendations for medical institutions are realized taking the user's emotional state into consideration.

[1511] "User's information processing device" refers to an electronic device used by a user, such as a smartphone, tablet, or personal computer, that has the function of collecting and transmitting voice data and digital internet information.

[1512] "Voice data" refers to data that is recorded in digital format via a microphone of what a user says to an information processing device.

[1513] "Digital Internet information" refers to historical data such as the search history a user has conducted on the Internet and the web pages they have accessed.

[1514] The "means for converting into text data" refers to a speech recognition technology or system for converting voice data into text format.

[1515] "Identifiers" are keywords or phrases extracted from the data being analyzed that are used to indicate specific health abnormalities.

[1516] "Medical facilities" are medical institutions such as hospitals and clinics that provide medical treatment according to the user's health condition.

[1517] An "emotion engine" is a technology or system that recognizes a user's emotional state based on their voice data and text data, and provides analysis results according to their emotions.

[1518] A "reservation system" is a system used by medical facilities to manage reservations, and has functions for accepting, managing, and confirming reservations.

[1519] A "notification" is a message sent from the server to the user's information processing device, and includes information such as a medical facility reservation suggestion, detailed reservation information, and a reminder.

[1520] "Emotional state" refers to the psychological state analyzed from the user's voice and text data, and indicates emotions such as joy, sadness, anger, and anxiety.

[1521] "Complementary analytics" refers to additional or detailed analytics based on a user's emotional state that can be used to make more accurate decisions or recommendations.

[1522] As an embodiment of the present invention, we have designed an automated system that collects voice data and digital internet information from a user's information processing device, analyzes this data to detect health abnormalities, recommends appropriate medical facilities, and makes appointments. This system also incorporates an emotion engine that recognizes the user's emotional state, allowing it to perform additional analysis and judgment based on the user's emotional state.

[1523] Overall overview

[1524] This system consists of the user's information processing device (smart device), a server, and the user's components. The user's information processing device collects voice data and digital internet information. The server analyzes this data, detects signs of health abnormalities, and recommends appropriate medical institutions. In addition, an emotion engine has been added to evaluate the user's emotional state and complement the analysis results of health abnormalities.

[1525] Hardware and software used

[1526] Information processing equipment: smart devices (smartphones, tablets, etc.)

[1527] Server: Dedicated server for data analysis

[1528] Speech recognition technology: Google Speech-to-Text API, etc.

[1529] NLP engines: spaCy and NLTK built in Python

[1530] Emotion recognition engine: Uses Praat and openSMILE as voice analysis tools

[1531] Geographic information service: Google Maps API

[1532] Push notification service: Firebase Cloud Messaging

[1533] Natural language description of the process

[1534] The device collects voice data when the user speaks to the smart device and the search history data they perform on the Internet. For example, if a user says, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "how to relieve insomnia," the device collects the search history.

[1535] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are then sent to a server. The server then uses natural language processing (NLP) technology to extract keywords that indicate health abnormalities. For example, if keywords such as "can't sleep well" or "insomnia" are detected, the server will determine that the user has sleep problems.

[1536] The server uses an emotion engine to recognize the user's emotional state from text and voice data. For example, if someone says, "I'm having a really hard time sleeping," the server can detect feelings of confusion and anxiety. This emotion assessment is used to complement the analysis results.

[1537] Based on the analysis results and emotion evaluation results, the server takes into account the user's location information and searches for appropriate nearby medical institutions. It then obtains available appointment dates and times from each medical institution and creates an appointment suggestion. The device displays the appointment suggestion notification sent from the server to the user. For example, a message may appear saying, "You seem to be having trouble sleeping. Would you like to make an appointment at a nearby sleep clinic?" The user can respond to this message with "Yes" or "No."

[1538] If the user responds "Yes," the server will link with the medical institution's reservation system to confirm the appointment for the specified date and time. After the appointment is successfully made, detailed information (date, time, location, name of medical institution, etc.) will be notified to the device. When the appointment date and time approaches, the server will generate a reminder notification and send it to the device. The device will display this reminder notification to the user to remind them not to forget to visit.

[1539] Specific examples

[1540] Example 1:

[1541] 1. The user says to their smartphone, "I haven't been sleeping well lately."

[1542] 2. The device converts the speech into text using a speech recognition function (for example, Google Speech-to-Text API) and sends the text data, such as "I haven't been sleeping well lately," to the server.

[1543] 3. At the same time, the user's browser search history data for "insomnia relief" is sent to the server.

[1544] 4. The server uses a Python NLP engine (e.g., spaCy) to analyze keywords such as "can't sleep well" and "insomnia" and also evaluate the user's emotional state.

[1545] 5. Based on the analysis results, use the Google Maps API to search for nearby sleep clinics and obtain available appointment dates and times.

[1546] 6. Using Firebase Cloud Messaging, display a notification on the device asking, "Would you like to make an appointment with a nearby sleep clinic?"

[1547] 7. If the user answers "Yes," the server calls the medical institution's reservation API to confirm the appointment with the specified sleep clinic.

[1548] 8. The device will display detailed information such as "Your appointment has been made at the sleep clinic on XX / XX / XX at XX time." A reminder notification will also be scheduled.

[1549] Prompt Sentence Examples

[1550] Please explain the process of suggesting an appointment for a sleep clinic to a user who says, "I haven't been sleeping well lately."

[1551] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1552] Step 1:

[1553] The device collects voice data when the user speaks to the smart device and the search history data of the user's internet searches. For example, if a user says to their smartphone, "I haven't been sleeping well lately," the device records the voice data. At the same time, if the user searches for keywords such as "insomnia relief," the device collects the search history.

[1554] Input: User voice data, internet search history data

[1555] Output: Recorded voice data, collected search history data

[1556] Step 2:

[1557] The collected voice data is converted into text data using the device's voice recognition function. For example, the Google Speech-to-Text API is used to convert the speech "I haven't been sleeping well lately" into text. The converted text data and search history data are then sent to the server.

[1558] Input: Recorded audio data

[1559] Output: Text data converted to character data, search history data

[1560] Step 3:

[1561] The server analyzes the text data and search history data sent to it. It uses natural language processing (NLP) technology, such as a Python NLP engine (spaCy or NLTK), to extract keywords that indicate health abnormalities. For example, it could extract keywords such as "not sleeping well" or "insomnia" and determine that the user has sleep problems.

[1562] Input: Text data converted to character data, search history data

[1563] Output: Extracted keywords related to health abnormalities

[1564] Step 4:

[1565] The server uses an emotion engine to recognize the user's emotional state from text and voice data. The emotion engine uses speech analysis tools (Praat and openSMILE) to analyze the intonation, speed, and volume of the voice data and extract emotions from the text data. For example, if a user says, "I'm having a really hard time sleeping," the emotion engine will detect confusion and anxiety.

[1566] Input: Text data converted to character data, audio data

[1567] Output: Data about the user's emotional state

[1568] Step 5:

[1569] Based on the analysis results and emotion evaluation results, the server searches for appropriate nearby medical institutions, taking into account the user's location information. For example, it uses the Google Maps API to search for nearby sleep clinics based on the user's location information and obtains available appointment dates and times at each medical institution.

[1570] Input: location information, data analysis results, emotion evaluation results

[1571] Output: List of suitable medical institutions, available appointment dates and times

[1572] Step 6:

[1573] The device displays the suggestions for available medical institutions sent from the server as a notification to the user. For example, it uses Firebase Cloud Messaging to notify the user with a message such as, "You seem to be having trouble sleeping. Would you like to make an appointment with a nearby sleep clinic?"

[1574] Input: List of suitable medical institutions, available appointment dates and times

[1575] Output: Notification of booking proposal to user

[1576] Step 7:

[1577] If the user accepts the reservation proposal, the server cooperates with the medical institution's reservation system to confirm the reservation at the specified date and time, for example, through the medical institution's reservation API.

[1578] Input: User response

[1579] Output: Confirmed reservation information

[1580] Step 8:

[1581] After the reservation is successfully made, the details (date, time, location, name of medical institution, etc.) are notified to the device. In addition, when the reservation date and time approaches, the server generates a reminder notification and sends it to the device. The device displays this reminder notification to the user.

[1582] Input: Confirmed reservation information, schedule information for reminder notifications

[1583] Output: Reservation details notification, reminder notification

[1584] (Application example 2)

[1585] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1586] Conventional health management systems and food delivery services have been unable to effectively utilize users' voice data and search history data to detect health abnormalities or recommend appropriate medical institutions. Furthermore, they have not made meal suggestions that take into account the user's emotional state, making it difficult for users to select meals based on their own health condition and emotions. This has led to the issue of users being unable to receive appropriate support to maintain and improve their health.

[1587] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1588] In this invention, the server includes means for collecting voice data and search history data from the user's information processing device, means for converting the voice data into text data, means for analyzing the text data and search history data and extracting keywords indicating health abnormalities, means for identifying appropriate medical institutions based on the keywords and obtaining available appointment dates and times, means for suggesting appropriate meals based on the user's health abnormalities and emotional state, and means for transmitting the contents of the suggested meals to the information processing device. This allows the user to receive meal suggestions based on their health and emotional state, and also enables them to quickly make appointments at necessary medical institutions.

[1589] An "information processing device" is an electronic device used by a user, such as a smartphone, tablet, or computer, and is a device for collecting and analyzing voice data and search history data.

[1590] "Voice data" refers to digital data that records what a user says to an information processing device.

[1591] "Search history data" refers to data that records the history of a user's internet searches.

[1592] "Text data" refers to data obtained by converting voice data into character information.

[1593] "Keywords" are important words that indicate health abnormalities or emotional states, extracted from text data and search history data.

[1594] A "medical institution" is an institution that provides medical services, such as a hospital or clinic.

[1595] An "emotion engine" is software that analyzes a user's emotional state from their voice and text data.

[1596] "Meal suggestions" are information that recommends optimal meals based on the user's health and emotional state.

[1597] A "notification" is a message or an alert sent to an information processing device.

[1598] A "reservation system" is a system for managing and processing reservations at medical institutions.

[1599] As an embodiment of the present invention, a system applied to a food delivery service consisting of a user, a terminal, and a server will be described. The operation of each element and the overall flow will be described in detail below.

[1600] Configuration and Operation

[1601] Data collection

[1602] The device collects voice data entered by the user using the voice recognition function. For example, if the user says, "I've been feeling tired lately," the device records the voice and saves it as voice data. At the same time, it collects the keywords the user searched for (e.g., "how to relieve fatigue") as search history data.

[1603] Data Conversion and Transmission

[1604] The collected voice data is converted into text data using the device's voice recognition function. This converted text data and search history data are sent to the server. For voice recognition, the speech_recognition library, for example, is used.

[1605] Data analysis

[1606] The server uses natural language processing (NLP) technology to analyze the submitted text data and search history data, extracting keywords that indicate health abnormalities (e.g., "fatigue" and "easily tired"). The analysis uses a health API (e.g., https: / / myhealthapi.example.com).

[1607] Emotion recognition and evaluation

[1608] The server identifies the user's emotional state using an emotion engine. This emotion engine evaluates the user's emotion from text data and voice tone. For emotion analysis, an emotion analysis API (e.g., https: / / myfeelingsapi.example.com) is used.

[1609] Meal suggestions

[1610] Based on the analyzed health and emotion data, the server suggests appropriate meals using a meal suggestion API (e.g., https: / / mydeliveryapi.example.com).

[1611] User Notification

[1612] The device will notify the user of the meal suggestions sent from the server, for example, by displaying a message saying, "We will suggest nutritious meals that suit your condition."

[1613] Specific examples

[1614] For example, if a user says, "I've been feeling a bit tired lately," the system analyzes the voice data and determines that the user is tired. Next, it performs emotion analysis to evaluate the user's emotions. Based on this, it suggests nutritious meals (e.g., vitamin-rich meals) and notifies the information processing device so that the user can immediately order the meals.

[1615] Prompt Sentence Examples

[1616] "User voice data: 'I've been feeling a bit tired lately.' Search history: 'How to relieve fatigue.' Analyze this data to suggest nutritious meals that are suitable for the user."

[1617] In this way, users can receive optimal support based on their health and emotional state.

[1618] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1619] Step 1:

[1620] The device collects voice data entered by the user using the voice recognition function. The input requires the user's voice, such as saying, "I've been feeling tired lately." The voice data is stored as digital data on the device. In addition, if the user enters search keywords such as "how to relieve fatigue," the device also collects search history data.

[1621] Step 2:

[1622] The device converts the collected voice data into text data using a voice recognition function. Digital voice data is used as input, and the conversion is performed using, for example, the speech_recognition library. The output is string data that can be analyzed from the voice data. At the same time, the converted text data and search history data are sent to the server.

[1623] Step 3:

[1624] The server analyzes the submitted text data and search history data using natural language processing (NLP) techniques. The input is the text data and search history data, and keywords (e.g., "fatigue" and "easily tired") are extracted using a health API (e.g., https: / / myhealthapi.example.com). The extracted keywords are obtained as output.

[1625] Step 4:

[1626] The server uses an emotion engine to analyze the user's emotions from the text data. It uses the text data obtained in step 3 as input and utilizes an emotion analysis API (e.g., https: / / myfeelingsapi.example.com). It outputs the user's emotional state (e.g., "stressed" or "tired").

[1627] Step 5:

[1628] The server suggests appropriate meals based on the health and emotional state data. As input, it uses keywords indicating health abnormalities and the emotional state data and sends a request to the meal suggestion API (e.g., https: / / mydeliveryapi.example.com). The output is a suggested meal menu.

[1629] Step 6:

[1630] The device notifies the user of the meal suggestions sent from the server. As input, it receives the suggestion data from the server and displays a message to the user saying, "We will suggest nutritious meals that suit your condition." The output is a notification message that the user can see.

[1631] This allows users to receive appropriate health support.

[1632] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1633] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1634] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1635] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1636] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1637] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1638] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1639] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1640] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1641] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1642] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1643] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1644] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1645] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1646] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1647] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1648] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1649] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1650] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1651] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1652] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1653] The following is further disclosed regarding the above embodiment.

[1654] (Claim 1)

[1655] A means for collecting voice data and search history data from a user's smart device;

[1656] means for converting the voice data into text data;

[1657] means for analyzing the text data and search history data and extracting keywords indicating health abnormalities;

[1658] A means for identifying an appropriate medical institution based on the keyword and acquiring an available reservation date and time;

[1659] means for sending notifications containing booking suggestions to the user's smart device;

[1660] a means for confirming the reservation in cooperation with a reservation system of the medical institution when the user accepts the reservation proposal;

[1661] means for notifying the user of the reservation details on a smart device;

[1662] A system including:

[1663] (Claim 2)

[1664] The system according to claim 1, further comprising means for identifying a medical institution based on the user's location information when the user's abnormal health data is confirmed.

[1665] (Claim 3)

[1666] 2. The system according to claim 1, further comprising means for sending a reminder notice before the reservation due date after the reservation is confirmed.

[1667] "Example 1"

[1668] (Claim 1)

[1669] A means for collecting voice data and search history data from a user's electronic device;

[1670] means for converting the voice data into text data;

[1671] means for analyzing the text data and search history data and extracting keywords indicating health abnormalities;

[1672] A means for identifying an appropriate medical institution based on the keyword and acquiring an available reservation date and time;

[1673] means for sending a notification containing a booking offer to the user's electronic device;

[1674] a means for confirming the reservation in cooperation with a reservation system of the medical institution when the user accepts the reservation proposal;

[1675] means for notifying the user of the reservation details on an electronic terminal;

[1676] means for generating and sending reminder notifications as the appointment date and time approaches;

[1677] A system including:

[1678] (Claim 2)

[1679] The system according to claim 1, further comprising means for identifying a medical institution based on location information of the user when a health abnormality is detected based on the keyword.

[1680] (Claim 3)

[1681] 2. The system according to claim 1, further comprising means for sending a reminder notice before the reservation due date after the reservation is confirmed.

[1682] "Application Example 1"

[1683] (Claim 1)

[1684] A means for collecting voice data and search history data from a user's computer terminal;

[1685] means for converting the voice data into character data;

[1686] means for analyzing the character data and search history data to extract words or phrases indicating abnormal health;

[1687] A means for identifying an appropriate medical facility and obtaining an available appointment date and time based on the phrase;

[1688] means for sending a notification containing a reservation suggestion to the user's computer terminal;

[1689] a means for confirming the reservation in cooperation with a reservation system of the medical facility when the user accepts the reservation proposal;

[1690] means for notifying the user of the reservation details on a computer terminal;

[1691] A means of analyzing users' health status data and suggesting health-related products;

[1692] A system including:

[1693] (Claim 2)

[1694] The system of claim 1, further comprising means for identifying a medical facility based on the user's location information when the user's abnormal health data is confirmed.

[1695] (Claim 3)

[1696] 2. The system according to claim 1, further comprising means for sending a reminder notice before the reservation due date after the reservation is confirmed.

[1697] "Example 2: Combining Emotion Engines"

[1698] (Claim 1)

[1699] means for collecting voice data and digital internet information from a user's information processing device;

[1700] means for converting the voice data into character data;

[1701] means for analyzing the character data and digital internet information and extracting an identifier indicating a health abnormality;

[1702] A means for identifying an appropriate medical facility and obtaining an available appointment date and time based on the identifier;

[1703] means for transmitting a notification including a reservation suggestion to the user's information processing device;

[1704] a means for confirming the reservation in cooperation with a reservation system of the medical facility when the user accepts the reservation proposal;

[1705] means for notifying the user's information processing device of detailed information about the reservation;

[1706] means including an emotion engine for recognizing the user's emotional state and performing complementary analysis based on the analysis results;

[1707] A system including:

[1708] (Claim 2)

[1709] 10. The system of claim 1, further comprising means for identifying a medical facility based on the user's geographic information when the user's abnormal health data is confirmed.

[1710] (Claim 3)

[1711] 2. The system according to claim 1, further comprising means for sending a reminder notice before the reservation due date after the reservation is confirmed.

[1712] "Application example 2 when combining emotion engines"

[1713] (Claim 1)

[1714] means for collecting ...

Claims

1. A means for collecting voice data and search history data from a user's smart device; means for converting the voice data into text data; means for analyzing the text data and search history data and extracting keywords indicating health abnormalities; A means for identifying an appropriate medical institution based on the keyword and acquiring an available reservation date and time; means for sending notifications containing booking suggestions to the user's smart device; a means for confirming the reservation in cooperation with a reservation system of the medical institution when the user accepts the reservation proposal; means for notifying the user of the reservation details on a smart device; A system including:

2. The system according to claim 1 , further comprising means for identifying a medical institution based on the user's location information when the user's abnormal health data is confirmed.

3. The system according to claim 1, further comprising means for sending a reminder notice before the reservation due date after the reservation is confirmed.

Images