System

Abstract

A system is provided.SOLUTION: A system comprising: means for converting an utterance of a doctor during a medical examination into a text in real time using a voice recognition technique; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting candidates of a disease name and a treatment policy from the generated electronic medical record; means for selecting an appropriate disease name and treatment policy from the presented disease names and treatment policies; and means for automatically generating a prescription based on the selected treatment policy.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional examination process, veterinarians had to manually record information obtained during the examination into a medical record, and then consider possible illnesses and treatment options. This meant that examinations took a long time, and the reliance on manual input made it prone to human error. The current situation, in particular, was problematic in that the recording of examination details and the determination of illnesses and treatment options were not efficient. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, a means is provided for converting a doctor's speech during a consultation into text in real time using speech recognition technology. Next, a means is provided for automatically generating an electronic medical record based on the converted text. Finally, a means is provided for automatically presenting disease names and treatment plan candidates from the generated electronic medical record. Furthermore, a means is provided for the doctor to select an appropriate disease name and treatment plan from the presented ones, and a means is provided for automatically generating a prescription based on the selected treatment plan. This automates the recording of consultation details and supports the determination of disease names and treatment plans, improving the efficiency of consultations and reducing manual recording errors.

[0006] "Voice recognition technology" is a technology that analyzes voice signals and converts them into text data.

[0007] "Doctor's remarks during examination" refers to audio information such as comments, instructions, and observations verbally given by the veterinarian during the course of the examination.

[0008] "Means for converting to text in real time" refers to a function that uses voice recognition technology to instantly convert a doctor's speech into text data.

[0009] An "electronic medical record" is a digital medical record that records, stores, and manages medical information electronically.

[0010] "Means for generating" is a function that automatically creates new information or files based on input data.

[0011] "Candidates for disease name and treatment plan" refers to diseases that are likely to be suspected based on the medical examination information and options for corresponding treatment methods.

[0012] "Means for automatic presentation" refers to a function in which the system automatically displays the name of the disease and treatment plan to the user based on the analysis results.

[0013] "Means for selection" refers to the interface or function that allows the user to select the most suitable option from the presented options.

[0014] A "prescription" is a document that lists the medications used to treat an illness and how to administer them.

[0015] "Means for automatic generation" refers to the function of the system to automatically create documents and information in a specified format based on the required data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function to automatically present the name of the disease and treatment plan. The operation of this system is described in detail below.

[0038] First, the user (veterinarian) starts the examination on the device. This can be done by pressing a button or using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and instructs it to start the device's voice recognition system.

[0039] Next, the user (veterinarian) comments while observing the animal's condition during the examination. For example, "This dog has lost its appetite recently and seems to be listless." The device records the veterinarian's comments in real time and converts the audio data into text. This converted text data is immediately sent to the server.

[0040] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0041] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while viewing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[0042] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0043] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information such as "lack of appetite" and "listlessness," and also presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[0044] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[0045] The processing flow will be explained below.

[0046] Step 1:

[0047] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[0048] Step 2:

[0049] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[0050] Step 3:

[0051] The server receives the consultation start signal and sends an instruction to start the voice recognition system to the terminal.

[0052] Step 4:

[0053] The device activates the voice recognition system and prepares to record and convert the veterinarian's speech into text in real time.

[0054] Step 5:

[0055] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[0056] Step 6:

[0057] The device records what the veterinarian says in real time and converts the audio data into text.

[0058] Step 7:

[0059] The terminal transmits the converted text data to the server in real time.

[0060] Step 8:

[0061] The server analyzes the received text data and initializes and generates the electronic medical record of the consultation.

[0062] Step 9:

[0063] The server records keywords and medical details based on the text data in the electronic medical record. Examples: "loss of appetite" and "lack of energy."

[0064] Step 10:

[0065] The server analyzes and generates possible disease names and treatment options based on the contents of the electronic medical record.

[0066] Step 11:

[0067] The server sends the generated disease name and treatment plan suggestions to the terminal and presents them to the user (veterinarian).

[0068] Step 12:

[0069] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[0070] Step 13:

[0071] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[0072] Step 14:

[0073] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[0074] Step 15:

[0075] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] In modern medical settings, doctors rely on manual input of information during consultations and the creation of prescriptions, resulting in problems such as long consultation times and a high risk of human error. Veterinarians, in particular, are required to accurately grasp the animal's condition, quickly record information, and provide an appropriate diagnosis and treatment plan. However, performing these tasks quickly and accurately is not easy, resulting in reduced consultation efficiency and problems with diagnostic accuracy. Given this background, there is a need for a system that can automate the consultation process and improve diagnostic accuracy.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes a means for converting the doctor's speech during the consultation into text in real time using speech recognition technology, a means for automatically generating an electronic medical record based on the converted text, and a means for automatically presenting disease names and treatment options from the generated electronic medical record. This enables quick and accurate recording of the contents of the consultation, improving diagnostic accuracy and shortening consultation times.

[0081] "Speech recognition technology" is a technology that analyzes voice data and converts it into corresponding text data.

[0082] "Statements made by a doctor during an examination" refers to statements made by a doctor during an examination to verbalize the patient's condition, diagnosis, treatment plan, etc.

[0083] "Converting to text in real time" refers to instantly generating corresponding text data from voice input.

[0084] "Electronic medical records" refers to a system that electronically records and manages information such as medical examination details, patient conditions, diagnosis results, and treatment plans.

[0085] "Automatically generated" refers to a process in which the system automatically generates data using a dedicated algorithm or program, without the need for manual operation.

[0086] "Candidate disease names and treatment options" refers to disease names and treatment options that may be considered based on the patient's symptoms and examination results.

[0087] "Presenting" refers to the system showing the analysis results and generated information to the user.

[0088] The "operation indicating the start of a consultation" refers to an operation such as pressing a button or issuing a voice command performed by the user to notify the system that a consultation has started.

[0089] "Natural language processing technology" refers to the technology of analyzing natural language (human language) using a computer, and extracting and processing semantic information from text data.

[0090] "Symptom analysis" refers to the process of identifying specific symptoms from data entered during the consultation or recorded text, and then determining a diagnosis and treatment plan based on that information.

[0091] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function for automatically presenting the name of the disease and a treatment plan. An embodiment of this system is described in detail below.

[0092] First, the user (veterinarian) performs an operation on the device to start the examination. This can be done by pressing a specific button or by entering the voice command "Start examination." The device detects this operation and notifies the server that the examination has begun. The server receives this notification and instructs the device to start a voice recognition system. Examples of voice recognition systems that can be used include Google Speech-to-Text API and Nuance Dragon.

[0093] Next, the user (veterinarian) speaks while observing the animal's condition during the examination. For example, consider the case where the user says, "This dog has had no appetite recently and seems to be listless." The device records the veterinarian's speech in real time, sends the voice data to a speech recognition system, and converts it into text. This converted text data is immediately sent to the server.

[0094] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0095] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while observing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[0096] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0097] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information about "lack of appetite" and "listlessness," and then presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[0098] An example of a prompt is as follows:

[0099] Start the consultation

[0100] This dog has no appetite and is not energetic.

[0101] What is the possible name of the disease?

[0102] Generate a prescription.

[0103] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[0104] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0105] Step 1:

[0106] Notification of start of consultation

[0107] User:

[0108] To start the consultation, press a specific button on the device or enter the voice command "Start consultation."

[0109] Input: Press a button or voice command "Start consultation"

[0110] Output: Notification signal for start of consultation

[0111] Device:

[0112] The operation to start a medical examination is detected, and a request to notify the server of the start of the medical examination is sent.

[0113] Input: Consultation start notification signal

[0114] Output: Request to start consultation to the server

[0115] Step 2:

[0116] Activating voice recognition

[0117] server:

[0118] A request to start a consultation is received and an instruction is given to the terminal to start the voice recognition system.

[0119] Input: Consultation start request

[0120] Output: Voice recognition system activation command

[0121] Device:

[0122] The voice recognition system is started upon receiving instructions from the server.

[0123] Input: Voice recognition system activation command

[0124] Output: Start the voice recognition system

[0125] Step 3:

[0126] Audio recording and text conversion

[0127] User:

[0128] While observing the animal's condition, the examiner will comment on the examination results.

[0129] Input: Animal condition observations

[0130] Output: Statement of medical examination details

[0131] Device:

[0132] The user's speech is recorded in real time, and the speech data is sent to a speech recognition system and converted into text data.

[0133] Input: Speech about medical examination (audio data)

[0134] Output: Converted text data (diagnosis details)

[0135] Step 4:

[0136] Transmitting text data and generating electronic medical records

[0137] Device:

[0138] The converted text data is sent to the server.

[0139] Input: Converted text data

[0140] Output: Send text data to the server

[0141] server:

[0142] The received text data is analyzed and an electronic medical record is automatically generated.

[0143] Extract keywords and medical details from the audio text and add them to the electronic record.

[0144] Input: Received text data

[0145] Output: Generated electronic medical record

[0146] Step 5:

[0147] Presentation of disease name and treatment plan

[0148] server:

[0149] Analysis is performed based on electronic medical records to automatically generate possible disease names and treatment options.

[0150] The generated disease name and treatment plan candidates are sent to the terminal.

[0151] Input: Electronic Medical Record

[0152] Output: Disease name and treatment options

[0153] Device:

[0154] The received disease name and treatment plan suggestions are presented to the user on the interface.

[0155] Input: Disease name and treatment options

[0156] Output: Presented disease name and treatment options

[0157] Step 6:

[0158] Automatically generate and save prescriptions

[0159] User:

[0160] Select the appropriate disease name and treatment plan from the presented options.

[0161] Input: Disease name and treatment options

[0162] Output: Selected treatment strategy

[0163] Device:

[0164] A prescription is automatically generated based on the selected treatment plan and sent to the server.

[0165] Input: Selected treatment plan

[0166] Output: Generated prescription

[0167] server:

[0168] The generated prescription is stored in a database.

[0169] Input: Generated prescription

[0170] Output: Saved prescription

[0171] Through the above steps, this system records and automatically processes examination details in real time, improving diagnostic accuracy and examination efficiency.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] In the traditional medical consultation process, doctors have to manually record the details of the consultation and select the diagnosis and treatment plan, which requires a great deal of time and effort. In addition, when dealing with customers in physical stores, it takes time and effort for store clerks to verbally answer customers' questions and provide product information. This can lead to a lack of efficiency and accuracy.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes means for converting speech made by a doctor during an examination into text in real time using speech recognition technology, means for automatically generating an electronic medical record based on the converted text, means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record, means for the doctor to select an appropriate disease name and treatment plan from the presented ones, means for automatically generating a prescription based on the selected treatment plan, and means for converting conversations with customers in a physical store into text in real time using speech recognition technology and presenting product information and recommended products based on the conversation. This enables the efficiency and accuracy of the doctor's examination process, and enables more efficient customer service and improved customer satisfaction in physical stores.

[0177] "Speech recognition technology" is a technology that converts speech into text.

[0178] "Doctor's statements during consultation" refers to audio information uttered by the doctor during the consultation.

[0179] "Means for converting into text in real time" refers to devices or systems that have the ability to instantly convert speech into text using speech recognition technology.

[0180] An "electronic medical record" refers to a database that records and manages a patient's medical information in digital format.

[0181] "Means of automatic generation" refers to devices or software that have the ability to generate data or information without human intervention.

[0182] "Candidate disease names and treatment plans" refers to information that presents multiple possible disease names and treatment methods based on the contents of the examination.

[0183] "Means of presentation" refers to devices or software that have the function of displaying or providing information through a system, such as a screen or audio.

[0184] "Means for selecting the appropriate one" refers to devices or software that have an interface or function for selecting the most appropriate one from the presented candidates.

[0185] "Prescription" refers to a document that contains instructions for prescribing medication to a patient.

[0186] "Brick-and-mortar customers" refers to consumers who visit a physical sales location.

[0187] "Means of converting conversations into text in real time" refers to a speech recognition system that instantly converts conversations with customers into text.

[0188] "Means for displaying product information and recommended products" refers to devices or software that have the function of displaying related products or recommended products based on the customer's inquiry.

[0189] A "generative AI model" refers to a model for generating natural language text data using artificial intelligence technology.

[0190] A "prompt sentence" refers to the text data input to a generative AI model.

[0191]

[0192] The system of this invention converts doctor's speech during consultations into text in real time, and aims to not only automatically generate electronic medical records and prescriptions, but also to improve the efficiency of customer service in brick-and-mortar stores. This system is realized using speech recognition technology, natural language processing technology, and generative AI models.

[0193] The server first converts the doctor's speech during the consultation into text in real time using speech recognition technology. To do this, it records the speech using a microphone and converts the recorded speech data into text using the Python speech_recognition library. The converted text data is immediately sent to the server.

[0194] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are recorded in the electronic medical record. Next, natural language processing technology is used to generate disease names and treatment plan candidates based on the electronic medical record, and these are presented to the user, the doctor. The presentation is done through an interface, and the doctor selects the appropriate candidate. Based on this, the server automatically generates and saves a prescription.

[0195] Next, we will explain customer service in physical stores. The server converts conversations with customers in physical stores into text in real time using speech recognition technology. To do this, it also uses a microphone to record the voice and converts it into text using the speech_recognition library. The converted text is immediately used as a prompt for the generative AI model.

[0196] The generative AI model generates product information and recommendations based on a prompt provided in text format, using OpenAI's GPT-3 API and inputting the following prompt:

[0197] If a customer says 'Is this shampoo suitable for dry skin?' provide the following information:

[0198] The generated answer might be something like, "Yes, this shampoo is suitable for dry skin. It contains moisturizing ingredients and keeps the skin moist." The server uses the pyttsx3 library to output this generated text as speech again.

[0199] This system automatically records and analyzes medical examination details, and by streamlining customer service in physical stores, it is possible to reduce the burden on doctors and improve customer satisfaction.

[0200] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0201] Step 1:

[0202] The user (doctor or store clerk) starts the voice recognition system. As input, the user performs an operation (pressing a button or issuing a voice command) to start voice recognition. As output, the voice recognition system starts and goes into a voice recording standby state.

[0203] Step 2:

[0204] The user's (doctor or store clerk's) speech is recorded by the terminal's microphone. As input, the user's spoken voice data is input to the microphone. As output, the voice data is recorded and saved in binary format.

[0205] Step 3:

[0206] The device converts the recorded voice data into text data in real time. This process uses the speech_recognition library. The input is the recorded voice data. The output is the voice data converted into text data.

[0207] Step 4:

[0208] The converted text data is sent from the terminal to the server. As input, the text data is sent from the terminal. As output, the server receives the text data.

[0209] Step 5:

[0210] The server generates an electronic medical record based on the received text data. The text data is input to the server. The output is an electronic medical record that records the details of the examination and keywords.

[0211] Step 6:

[0212] The server generates disease names and treatment plan candidates based on the electronic medical records and presents them to the user (doctor). The input is the electronic medical record data entered into the server. The output is a list of disease names and treatment plan candidates that are displayed to the user.

[0213] Step 7:

[0214] The user (doctor) selects the appropriate disease name and treatment plan from the presented list. As input, a list of disease names and treatment plans is presented to the user. As output, the disease name and treatment plan selected by the user are sent to the server.

[0215] Step 8:

[0216] The server automatically generates and stores a prescription based on the selected treatment plan. As input, the selected treatment plan is entered into the server. As output, a prescription is generated and stored.

[0217] Step 9:

[0218] When the device returns to serving customers in a physical store, it reactivates the voice recognition system. The input is the customer's question or statement, which is entered into the microphone. The output is a recording of the speech and converted into text data.

[0219] Step 10:

[0220] The terminal sends the converted text data to the server as a prompt sentence for the generative AI model. As input, text data containing the customer's utterance is generated. As output, the generated prompt sentence is input to the server.

[0221] Step 11:

[0222] The server uses a generative AI model to generate a response text based on the prompt sentence. This process uses the OpenAI GPT-3 API. The prompt sentence is entered into the API as input. The response text is generated as output.

[0223] Step 12:

[0224] The server sends the generated answer text to the terminal, which outputs it as voice using the pyttsx3 library. As input, the answer text is input from the server. As output, the text is converted into voice so that the user (store clerk) can provide the answer to the customer.

[0225] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0226] The present invention is a system for streamlining the examination process for veterinarians and improving the accuracy of diagnoses, and by combining it with an emotion engine, the quality of examinations is further improved. Specific embodiments of the present invention are described below.

[0227] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or by using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and sends instructions to activate the device's voice recognition system and emotion engine.

[0228] Next, the user (veterinarian) examines the animal and verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. This text data is immediately sent to the server.

[0229] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0230] Furthermore, the emotion engine analyzes the content and voice patterns of the veterinarian's speech to recognize their emotions. For example, it can detect when the veterinarian is anxious or suspicious. The emotional information recognized by the emotion engine is added to the text data and recorded in the electronic medical record.

[0231] The server then automatically analyzes possible illnesses and treatment options based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal. The presented options are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The veterinarian selects the most appropriate option from the presented options and explains it to the owner.

[0232] For example, if the emotion engine detects a veterinarian's anxiety, it will adjust the priorities of potential diagnoses and treatment options to match that anxiety, allowing veterinarians to access the information they need more quickly and improving the accuracy of their diagnoses.

[0233] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0234] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[0235] In this way, the system of the present invention automates the recording of medical examination details and supports appropriate diagnosis and treatment based on emotional information, thereby improving the efficiency and accuracy of medical examinations.

[0236] The processing flow will be explained below.

[0237] Step 1:

[0238] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[0239] Step 2:

[0240] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[0241] Step 3:

[0242] The server receives the consultation start signal and sends an instruction to activate the voice recognition system and emotion engine to the terminal.

[0243] Step 4:

[0244] The device activates the voice recognition system and emotion engine, ready to record and transcribe the veterinarian's speech in real time.

[0245] Step 5:

[0246] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[0247] Step 6:

[0248] The device records the veterinarian's speech in real time, converts the audio data into text, and analyzes the emotions expressed in the speech using an emotion engine.

[0249] Step 7:

[0250] The device transmits the converted text data and analyzed emotional information to the server in real time.

[0251] Step 8:

[0252] The server analyzes the received text data and emotional information, and initializes and generates an electronic medical record of the consultation.

[0253] Step 9:

[0254] The server records keywords and medical examination details based on text data in the electronic medical record. At the same time, the emotion engine adds emotional information detected by the server to the medical record. Examples: "loss of appetite," "lack of energy," "doctor's concern."

[0255] Step 10:

[0256] The server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal.

[0257] Step 11:

[0258] The terminal receives the disease name and treatment options from the server and displays them to the veterinarian. The priority of the options may be changed based on the emotion information detected by the emotion engine.

[0259] Step 12:

[0260] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[0261] Step 13:

[0262] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[0263] Step 14:

[0264] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[0265] Step 15:

[0266] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[0267] In this way, by combining the emotion engine, it is possible to present an appropriate diagnosis and treatment plan that takes into account the examination information and the veterinarian's emotions, further improving the efficiency and accuracy of examinations.

[0268] Example 2

[0269] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0270] In conventional veterinary consultations, recording the details of the examination and improving the accuracy of the diagnosis have been issues. In particular, when veterinarians feel anxious or impatient, their emotions can affect the diagnosis. Furthermore, when the details of the examination are recorded manually, there is a high risk of errors or omissions. In addition, presenting possible diagnoses and selecting treatment plans is complicated, making it difficult to achieve an efficient examination process.

[0271] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for converting the veterinarian's remarks during the examination into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the veterinarian's remarks and voice patterns and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on emotion information. This enables the automatic recording of the examination details and the presentation of diagnosis candidates taking into account the veterinarian's emotion information.

[0272] "Speech recognition technology" is a technology that converts the voice spoken by a user into digital text in real time.

[0273] An "electronic medical record" is a database for digitally storing and managing the details of medical examinations, symptoms, diagnostic information, and treatment plans recorded during medical examinations.

[0274] The "means for automatically presenting disease names and treatment plan candidates" is a system that presents possible disease names and appropriate treatment plans to the user in list form based on the contents of the examination.

[0275] The "emotion engine" is a technology that analyzes the content and voice patterns of veterinarians' statements to recognize their emotions and generate emotional information.

[0276] "Emotional information" is data that represents the emotional state of the veterinarian and is used to adjust the priorities of candidate diagnoses.

[0277] The "means for adjusting the priority of candidate diagnoses" is an algorithm that changes the order of presented disease names and treatment plans, taking into account the veterinarian's emotional information.

[0278] "Means for converting into text in real time" refers to a function that instantly converts what the veterinarian says into text format and sends that text to the system.

[0279] An "operation indicating the start of examination" is a specific action taken by the veterinarian to start the examination, and includes, for example, pressing a button or inputting a voice command.

[0280] The present invention provides a system for streamlining the examination process by veterinarians and improving the accuracy of diagnoses. The system includes: means for converting speech from a veterinarian during an examination into text in real time using speech recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the content and speech patterns of the veterinarian's speech and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on the emotion information.

[0281] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or using a specific voice command. When the device detects this operation, it sends a notification that the examination has started to the server. The server then sends instructions to the device to start the voice recognition system and emotion engine.

[0282] While examining an animal, the user (veterinarian) verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. The converted text data is immediately sent to the server.

[0283] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded. Furthermore, the emotion engine analyzes the content of the veterinarian's speech and voice patterns to recognize the veterinarian's emotions. For example, it can detect when the veterinarian is anxious or suspicious. This emotional information is added to the text data and recorded in the electronic medical record.

[0284] Next, the server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents the candidates on the terminal. The presented candidates are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The user (veterinarian) selects the most appropriate one from the presented candidates and explains it to the owner. If the emotion engine detects the veterinarian's anxiety, the priority of the disease name and treatment plan candidates is adjusted to match that anxiety. This allows veterinarians to access the information they need quickly, improving the accuracy of diagnoses.

[0285] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0286] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[0287] An example prompt might be: "Please explain in detail, step by step, how your system would respond when the veterinarian says, 'This dog has no appetite and is listless.'"

[0288] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0289] Step 1:

[0290] The user (veterinarian) starts the examination on the terminal.

[0291] Input: Pressing the Start Visit button or a specific voice command.

[0292] Specific operation: The user presses the "Start consultation" button on the device's touchscreen or says the voice command "Start consultation."

[0293] Output: Consultation start event.

[0294] Step 2:

[0295] The terminal detects the start of a medical examination event and transmits a notice of the start of a medical examination to the server.

[0296] Input: Consultation start event.

[0297] Specific operation: The device sends an HTTP request to the server triggered by the start of a consultation event, which includes metadata such as the timestamp of the consultation start and the veterinarian's ID.

[0298] Output: Notification to the server that consultation has started.

[0299] Step 3:

[0300] The server receives notification that the consultation has started and sends instructions to the terminal to activate the voice recognition system and emotion engine.

[0301] Input: Notification of start of consultation from terminal.

[0302] Specific operation: As an HTTP response, the server sends a message to the terminal containing commands to start the voice recognition system and emotion engine.

[0303] Output: Instructions to the device to start the voice recognition system and emotion engine.

[0304] Step 4:

[0305] The user (veterinarian) examines the animal and verbally explains the animal's condition and observation results.

[0306] Input: Veterinarian statement.

[0307] Specific actions: The user (veterinarian) verbally explains the results of the examination, such as, "This dog has had no appetite recently and seems to be in a bad mood."

[0308] Output: The veterinarian's remarks are recorded as audio data.

[0309] Step 5:

[0310] The device records what the veterinarian says as audio data in real time and converts the audio data into text.

[0311] Input: Veterinarian speech data.

[0312] How it works: A speech recognition system converts speech data into text using machine learning models.

[0313] Output: Text data.

[0314] Step 6:

[0315] The terminal transmits the generated text data to the server.

[0316] Input: Text data.

[0317] Specific operation: The generated text data is packaged in JSON format and sent to the server as an HTTP request.

[0318] Output: Sends text data to the server.

[0319] Step 7:

[0320] The server generates an electronic medical record based on the received text data.

[0321] Input: Text data.

[0322] Specific operation: The server analyzes the received text data, identifies important medical details using a keyword extraction algorithm, and records them in the electronic medical record.

[0323] Output: Generated electronic medical record.

[0324] Step 8:

[0325] The server works in conjunction with the text data to run an emotion engine and analyze the veterinarian's emotions.

[0326] Input: Audio data.

[0327] How it works: The emotion engine uses machine learning models to analyze the tone, rate, emphasis, etc. of the voice and infer the veterinarian's emotional state.

[0328] Output: Emotional information.

[0329] Step 9:

[0330] Emotional information is added to the text data and recorded in the electronic medical record.

[0331] Input: Emotion information.

[0332] Specific operation: The veterinarian's emotion label is added to the text data and sent back to the server.

[0333] Output: Electronic medical record with emotional information added.

[0334] Step 10:

[0335] The server automatically analyzes the disease name and possible treatment options based on the contents of the electronic medical record and emotional information, and presents them on the terminal.

[0336] Input: Electronic medical record and emotional information.

[0337] How it works: The server analyzes the EMR and emotion information, uses machine learning models to search for relevant cases in the database, and sends a prioritized list of potential diagnoses and treatment options to the device.

[0338] Output: A list of potential diagnoses and treatment strategies.

[0339] Step 11:

[0340] The terminal displays the presented candidates in an interface that allows the user (veterinarian) to select one.

[0341] Input: A list of potential diagnoses and treatment strategies.

[0342] Specific operation: The terminal visually displays diagnostic candidates and treatment plans, and provides an interface that allows the user to easily make selections.

[0343] Output: User selection of potential diagnoses and treatment strategies.

[0344] Step 12:

[0345] The user (veterinarian) selects a candidate diagnosis, and the terminal automatically generates an appropriate prescription based on that.

[0346] Input: Selected candidate diagnoses.

[0347] Specific operation: The terminal generates a prescription based on the selected diagnostic candidate according to the type and size of the animal and stores it on the server.

[0348] Output: Auto-generated prescription.

[0349] (Application example 2)

[0350] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0351] The modern veterinarian consultation process requires accurate recording of consultation data, highly accurate diagnosis, and efficient treatment decisions. However, current methods require all of these processes to be done manually, which is not only time-consuming and labor-intensive, but also makes the accuracy of diagnosis highly dependent on the veterinarian's experience. Furthermore, as the demand for remote consultations increases, there is a need to provide consultations of the same quality remotely. To solve these challenges, a system that integrates speech recognition technology, emotion analysis technology, natural language processing technology, and remote consultation functionality is required.

[0352] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for converting the doctor's utterances during the consultation into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for analyzing the doctor's emotional state using emotion analysis technology and adding it to the text data; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record and emotion information; means for the doctor to select an appropriate disease name and treatment plan from the presented disease names and treatment plans; means for automatically generating a prescription based on the selected treatment plan; and means having a real-time video call function for remote consultations. This realizes efficient examination processes and improved diagnostic accuracy, making it possible to provide the same quality as conventional methods even in remote consultations.

[0353] "Speech recognition technology" is a technology that converts voice data into text data.

[0354] "Emotion analysis technology" is a technology that analyzes and determines a speaker's emotional state from voice and text data.

[0355] An "electronic medical record" is digital data used to manage the medical records of patients and animals on an information system, replacing paper medical records.

[0356] "Natural language processing technology" is a technology that allows computers to understand and process natural language.

[0357] "Remote consultation" is a method in which a doctor can conduct an examination via the Internet while being in a remote location from the patient or owner.

[0358] "Candidates for diagnosis and treatment" is a list of possible diagnoses and treatment methods based on the examination.

[0359] The "real-time video call function" is a function that allows two-way communication of video and audio in real time via the Internet.

[0360] A "prescription" is a document that lists specific medications or treatments prescribed by a physician for a patient or animal.

[0361] The present invention provides a system for improving the efficiency of the process when a veterinarian examines an animal and improving the accuracy of diagnosis. Specific embodiments are described below.

[0362] System configuration

[0363] 1. Use of voice recognition technology

[0364] The system uses speech recognition technology to convert what the veterinarian says into text in real time, using the Google Cloud Speech-to-Text API or a similar speech recognition API.

[0365] 2. Use of emotion analysis technology

[0366] Use IBM Watson Tone Analyzer or a similar sentiment analysis engine to analyze the veterinarian's speech and emotional state from their voice patterns.

[0367] 3. Generating Electronic Medical Records

[0368] Based on the converted text data and emotion information, an electronic medical record is automatically generated. The medical records are stored in a database system (e.g., MongoDB).

[0369] 4. Implementing real-time video calling

[0370] Incorporate video calling technology (e.g., WebRTC) to allow owners to receive consultations remotely.

[0371] 5. Diagnosis and treatment plan

[0372] Based on the generated electronic medical record and emotional information, symptoms are analyzed using natural language processing technology (e.g., generative AI model), and appropriate disease names and treatment options are generated and presented to the veterinarian.

[0373] 6. Automatic prescription generation

[0374] Based on the diagnostic results selected by the veterinarian, the appropriate prescription is automatically generated and provided to the owner in digital format.

[0375] Specific examples of functions

[0376] Process during consultation:

[0377] 1. Launching speech recognition and emotion analysis

[0378] When the veterinarian presses the "Start Examination" button, the system simultaneously activates voice recognition and emotion analysis technologies.

[0379] 2. Processing of veterinarians' speech data

[0380] When a veterinarian says, "This dog has been coughing a lot recently and seems lethargic," the voice recognition system converts this statement into text data in real time.

[0381] The emotion analysis engine detects emotions such as "worry" from the veterinarian's comments and adds them to the text data.

[0382] 3. Automatic generation of electronic medical records

[0383] Based on the converted text data and emotional information, content such as "Symptoms: cough, lethargy" and "Veterinarian's emotions: worried" are automatically recorded in the electronic medical record.

[0384] 4. Providing diagnosis and treatment guidelines

[0385] The system references electronic medical records and emotional information, and uses a "generative AI model" to suggest possible disease names such as "bronchitis" and "pneumonia," as well as treatment options.

[0386] The veterinarian will confirm the diagnosis based on the options presented.

[0387] 5. Automatic prescription generation

[0388] Based on the confirmed diagnosis, a specific prescription such as "antibiotics (300 mg) twice a day for 7 days" is automatically generated.

[0389] Example prompt sentence:

[0390] "Examination: This dog has been coughing a lot recently and seems lethargic.

[0391] Veterinarian's emotional state: Anxiety.

[0392] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[0393] In this way, the system supports veterinarians throughout the entire consultation process, enabling efficient and highly accurate consultations.

[0394] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0395] Step 1:

[0396] When the user (veterinarian) presses the button to start the consultation, the server simultaneously launches the speech recognition technology and emotion analysis technology. The input is the operation signal to start the consultation, and based on this, the server initializes the speech recognition API (e.g., Google Cloud Speech-to-Text) and emotion analysis engine (e.g., IBM Watson Tone Analyzer). The output is a state in which these technologies are ready to operate.

[0397] Step 2:

[0398] The device uses real-time video calling functionality to transmit video and audio between the owner and the veterinarian in two directions. The input is the audio and video data of the owner and the veterinarian, which is processed in real time through a video calling library (e.g., WebRTC). The output is a real-time video call session.

[0399] Step 3:

[0400] The device uses a voice recognition system to convert what the veterinarian says during the examination into text in real time. The input is the veterinarian's voice data, which is analyzed by the voice recognition API to generate text data. The output is the text data of the veterinarian's speech.

[0401] Step 4:

[0402] The server analyzes the text data using an emotion analysis engine to analyze the veterinarian's emotional state. The input is the text data of the veterinarian's comments, which the emotion analysis engine analyzes and adds emotional information. The output is text data with the added emotional information.

[0403] Step 5:

[0404] The server automatically generates an electronic medical record based on text data with added emotional information. The input is text data and emotional information, which are recorded in a database system (e.g., MongoDB) to generate the electronic medical record. The output is the electronic medical record data.

[0405] Step 6:

[0406] The server uses natural language processing technology to analyze symptoms using electronic medical record data and emotional information. The input is the electronic medical record data and emotional information, and by invoking the generative AI model with a prompt sentence, it generates appropriate disease names and treatment plan candidates. The output is disease names and treatment plan candidates.

[0407] Step 7:

[0408] The terminal presents the generated disease name and treatment plan candidate list to the veterinarian, allowing them to select the most appropriate diagnosis. The disease name and treatment plan candidate list are input, and the veterinarian makes the selection via the interface. The selected diagnosis result is output.

[0409] Step 8:

[0410] The server automatically generates an appropriate prescription based on the selected diagnostic results and provides it to the owner in digital form. The input is the selected diagnostic results, and the prescription data is generated based on this. The output is a digital prescription.

[0411] Specific actions

[0412] Example prompt sentence:

[0413] "Examination: This dog has been coughing a lot recently and seems lethargic.

[0414] Veterinarian's emotional state: Anxiety.

[0415] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[0416] Through this series of steps, the system streamlines the entire consultation process and enables highly accurate diagnosis and treatment.

[0417] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0418] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0419] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0420] [Second embodiment]

[0421] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0422] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0423] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0424] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0425] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0426] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0427] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0428] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0429] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0430] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0431] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0432] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0433] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function to automatically present the name of the disease and treatment plan. The operation of this system is described in detail below.

[0434] First, the user (veterinarian) starts the examination on the device. This can be done by pressing a button or using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and instructs it to start the device's voice recognition system.

[0435] Next, the user (veterinarian) comments while observing the animal's condition during the examination. For example, "This dog has lost its appetite recently and seems to be listless." The device records the veterinarian's comments in real time and converts the audio data into text. This converted text data is immediately sent to the server.

[0436] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0437] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while viewing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[0438] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0439] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information such as "lack of appetite" and "listlessness," and also presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[0440] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[0441] The processing flow will be explained below.

[0442] Step 1:

[0443] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[0444] Step 2:

[0445] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[0446] Step 3:

[0447] The server receives the consultation start signal and sends an instruction to start the voice recognition system to the terminal.

[0448] Step 4:

[0449] The device activates the voice recognition system and prepares to record and convert the veterinarian's speech into text in real time.

[0450] Step 5:

[0451] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[0452] Step 6:

[0453] The device records what the veterinarian says in real time and converts the audio data into text.

[0454] Step 7:

[0455] The terminal transmits the converted text data to the server in real time.

[0456] Step 8:

[0457] The server analyzes the received text data and initializes and generates the electronic medical record of the consultation.

[0458] Step 9:

[0459] The server records keywords and medical details based on the text data in the electronic medical record. Examples: "loss of appetite" and "lack of energy."

[0460] Step 10:

[0461] The server analyzes and generates possible disease names and treatment options based on the contents of the electronic medical record.

[0462] Step 11:

[0463] The server sends the generated disease name and treatment plan suggestions to the terminal and presents them to the user (veterinarian).

[0464] Step 12:

[0465] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[0466] Step 13:

[0467] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[0468] Step 14:

[0469] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[0470] Step 15:

[0471] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[0472] Example 1

[0473] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0474] In modern medical settings, doctors rely on manual input of information during consultations and the creation of prescriptions, resulting in problems such as long consultation times and a high risk of human error. Veterinarians, in particular, are required to accurately grasp the animal's condition, quickly record information, and provide an appropriate diagnosis and treatment plan. However, performing these tasks quickly and accurately is not easy, resulting in reduced consultation efficiency and problems with diagnostic accuracy. Given this background, there is a need for a system that can automate the consultation process and improve diagnostic accuracy.

[0475] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0476] In this invention, the server includes a means for converting the doctor's speech during the consultation into text in real time using speech recognition technology, a means for automatically generating an electronic medical record based on the converted text, and a means for automatically presenting disease names and treatment options from the generated electronic medical record. This enables quick and accurate recording of the contents of the consultation, improving diagnostic accuracy and shortening consultation times.

[0477] "Speech recognition technology" is a technology that analyzes voice data and converts it into corresponding text data.

[0478] "Statements made by a doctor during an examination" refers to statements made by a doctor during an examination to verbalize the patient's condition, diagnosis, treatment plan, etc.

[0479] "Converting to text in real time" refers to instantly generating corresponding text data from voice input.

[0480] "Electronic medical records" refers to a system that electronically records and manages information such as medical examination details, patient conditions, diagnosis results, and treatment plans.

[0481] "Automatically generated" refers to a process in which the system automatically generates data using a dedicated algorithm or program, without the need for manual operation.

[0482] "Candidate disease names and treatment options" refers to disease names and treatment options that may be considered based on the patient's symptoms and examination results.

[0483] "Presenting" refers to the system showing the analysis results and generated information to the user.

[0484] The "operation indicating the start of a consultation" refers to an operation such as pressing a button or issuing a voice command performed by the user to notify the system that a consultation has started.

[0485] "Natural language processing technology" refers to the technology of analyzing natural language (human language) using a computer, and extracting and processing semantic information from text data.

[0486] "Symptom analysis" refers to the process of identifying specific symptoms from data entered during the consultation or recorded text, and then determining a diagnosis and treatment plan based on that information.

[0487] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function for automatically presenting the name of the disease and a treatment plan. An embodiment of this system is described in detail below.

[0488] First, the user (veterinarian) performs an operation on the device to start the examination. This can be done by pressing a specific button or by entering the voice command "Start examination." The device detects this operation and notifies the server that the examination has begun. The server receives this notification and instructs the device to start a voice recognition system. Examples of voice recognition systems that can be used include Google Speech-to-Text API and Nuance Dragon.

[0489] Next, the user (veterinarian) speaks while observing the animal's condition during the examination. For example, consider the case where the user says, "This dog has had no appetite recently and seems to be listless." The device records the veterinarian's speech in real time, sends the voice data to a speech recognition system, and converts it into text. This converted text data is immediately sent to the server.

[0490] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0491] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while observing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[0492] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0493] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information about "lack of appetite" and "listlessness," and then presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[0494] An example of a prompt is as follows:

[0495] Start the consultation

[0496] This dog has no appetite and is not energetic.

[0497] What is the possible name of the disease?

[0498] Generate a prescription.

[0499] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[0500] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0501] Step 1:

[0502] Notification of start of consultation

[0503] User:

[0504] To start the consultation, press a specific button on the device or enter the voice command "Start consultation."

[0505] Input: Press a button or voice command "Start consultation"

[0506] Output: Notification signal for start of consultation

[0507] Device:

[0508] The operation to start a medical examination is detected, and a request to notify the server of the start of the medical examination is sent.

[0509] Input: Consultation start notification signal

[0510] Output: Request to start consultation to the server

[0511] Step 2:

[0512] Activating voice recognition

[0513] server:

[0514] A request to start a consultation is received and an instruction is given to the terminal to start the voice recognition system.

[0515] Input: Consultation start request

[0516] Output: Voice recognition system activation command

[0517] Device:

[0518] The voice recognition system is started upon receiving instructions from the server.

[0519] Input: Voice recognition system activation command

[0520] Output: Start the voice recognition system

[0521] Step 3:

[0522] Audio recording and text conversion

[0523] User:

[0524] While observing the animal's condition, the examiner will comment on the examination results.

[0525] Input: Animal condition observations

[0526] Output: Statement of medical examination details

[0527] Device:

[0528] The user's speech is recorded in real time, and the speech data is sent to a speech recognition system and converted into text data.

[0529] Input: Speech about medical examination (audio data)

[0530] Output: Converted text data (diagnosis details)

[0531] Step 4:

[0532] Transmitting text data and generating electronic medical records

[0533] Device:

[0534] The converted text data is sent to the server.

[0535] Input: Converted text data

[0536] Output: Send text data to the server

[0537] server:

[0538] The received text data is analyzed and an electronic medical record is automatically generated.

[0539] Extract keywords and medical details from the audio text and add them to the electronic record.

[0540] Input: Received text data

[0541] Output: Generated electronic medical record

[0542] Step 5:

[0543] Presentation of disease name and treatment plan

[0544] server:

[0545] Analysis is performed based on electronic medical records to automatically generate possible disease names and treatment options.

[0546] The generated disease name and treatment plan candidates are sent to the terminal.

[0547] Input: Electronic Medical Record

[0548] Output: Disease name and treatment options

[0549] Device:

[0550] The received disease name and treatment plan suggestions are presented to the user on the interface.

[0551] Input: Disease name and treatment options

[0552] Output: Presented disease name and treatment options

[0553] Step 6:

[0554] Automatically generate and save prescriptions

[0555] User:

[0556] Select the appropriate disease name and treatment plan from the presented options.

[0557] Input: Disease name and treatment options

[0558] Output: Selected treatment strategy

[0559] Device:

[0560] A prescription is automatically generated based on the selected treatment plan and sent to the server.

[0561] Input: Selected treatment plan

[0562] Output: Generated prescription

[0563] server:

[0564] The generated prescription is stored in a database.

[0565] Input: Generated prescription

[0566] Output: Saved prescription

[0567] Through the above steps, this system records and automatically processes examination details in real time, improving diagnostic accuracy and examination efficiency.

[0568] (Application example 1)

[0569] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0570] In the traditional medical consultation process, doctors have to manually record the details of the consultation and select the diagnosis and treatment plan, which requires a great deal of time and effort. In addition, when dealing with customers in physical stores, it takes time and effort for store clerks to verbally answer customers' questions and provide product information. This can lead to a lack of efficiency and accuracy.

[0571] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0572] In this invention, the server includes means for converting speech made by a doctor during an examination into text in real time using speech recognition technology, means for automatically generating an electronic medical record based on the converted text, means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record, means for the doctor to select an appropriate disease name and treatment plan from the presented ones, means for automatically generating a prescription based on the selected treatment plan, and means for converting conversations with customers in a physical store into text in real time using speech recognition technology and presenting product information and recommended products based on the conversation. This enables the efficiency and accuracy of the doctor's examination process, and enables more efficient customer service and improved customer satisfaction in physical stores.

[0573] "Speech recognition technology" is a technology that converts speech into text.

[0574] "Doctor's statements during consultation" refers to audio information uttered by the doctor during the consultation.

[0575] "Means for converting into text in real time" refers to devices or systems that have the ability to instantly convert speech into text using speech recognition technology.

[0576] An "electronic medical record" refers to a database that records and manages a patient's medical information in digital format.

[0577] "Means of automatic generation" refers to devices or software that have the ability to generate data or information without human intervention.

[0578] "Candidate disease names and treatment plans" refers to information that presents multiple possible disease names and treatment methods based on the contents of the examination.

[0579] "Means of presentation" refers to devices or software that have the function of displaying or providing information through a system, such as a screen or audio.

[0580] "Means for selecting the appropriate one" refers to devices or software that have an interface or function for selecting the most appropriate one from the presented candidates.

[0581] "Prescription" refers to a document that contains instructions for prescribing medication to a patient.

[0582] "Brick-and-mortar customers" refers to consumers who visit a physical sales location.

[0583] "Means of converting conversations into text in real time" refers to a speech recognition system that instantly converts conversations with customers into text.

[0584] "Means for displaying product information and recommended products" refers to devices or software that have the function of displaying related products or recommended products based on the customer's inquiry.

[0585] A "generative AI model" refers to a model for generating natural language text data using artificial intelligence technology.

[0586] A "prompt sentence" refers to the text data input to a generative AI model.

[0587]

[0588] The system of this invention converts doctor's speech during consultations into text in real time, and aims to not only automatically generate electronic medical records and prescriptions, but also to improve the efficiency of customer service in brick-and-mortar stores. This system is realized using speech recognition technology, natural language processing technology, and generative AI models.

[0589] The server first converts the doctor's speech during the consultation into text in real time using speech recognition technology. To do this, it records the speech using a microphone and converts the recorded speech data into text using the Python speech_recognition library. The converted text data is immediately sent to the server.

[0590] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are recorded in the electronic medical record. Next, natural language processing technology is used to generate disease names and treatment plan candidates based on the electronic medical record, and these are presented to the user, the doctor. The presentation is done through an interface, and the doctor selects the appropriate candidate. Based on this, the server automatically generates and saves a prescription.

[0591] Next, we will explain customer service in physical stores. The server converts conversations with customers in physical stores into text in real time using speech recognition technology. To do this, it also uses a microphone to record the voice and converts it into text using the speech_recognition library. The converted text is immediately used as a prompt for the generative AI model.

[0592] The generative AI model generates product information and recommendations based on a prompt provided in text format, using OpenAI's GPT-3 API and inputting the following prompt:

[0593] If a customer says 'Is this shampoo suitable for dry skin?' provide the following information:

[0594] The generated answer might be something like, "Yes, this shampoo is suitable for dry skin. It contains moisturizing ingredients and keeps the skin moist." The server uses the pyttsx3 library to output this generated text as speech again.

[0595] This system automatically records and analyzes medical examination details, and by streamlining customer service in physical stores, it is possible to reduce the burden on doctors and improve customer satisfaction.

[0596] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0597] Step 1:

[0598] The user (doctor or store clerk) starts the voice recognition system. As input, the user performs an operation (pressing a button or issuing a voice command) to start voice recognition. As output, the voice recognition system starts and goes into a voice recording standby state.

[0599] Step 2:

[0600] The user's (doctor or store clerk's) speech is recorded by the terminal's microphone. As input, the user's spoken voice data is input to the microphone. As output, the voice data is recorded and saved in binary format.

[0601] Step 3:

[0602] The device converts the recorded voice data into text data in real time. This process uses the speech_recognition library. The input is the recorded voice data. The output is the voice data converted into text data.

[0603] Step 4:

[0604] The converted text data is sent from the terminal to the server. As input, the text data is sent from the terminal. As output, the server receives the text data.

[0605] Step 5:

[0606] The server generates an electronic medical record based on the received text data. The text data is input to the server. The output is an electronic medical record that records the details of the examination and keywords.

[0607] Step 6:

[0608] The server generates disease names and treatment plan candidates based on the electronic medical records and presents them to the user (doctor). The input is the electronic medical record data entered into the server. The output is a list of disease names and treatment plan candidates that are displayed to the user.

[0609] Step 7:

[0610] The user (doctor) selects the appropriate disease name and treatment plan from the presented list. As input, a list of disease names and treatment plans is presented to the user. As output, the disease name and treatment plan selected by the user are sent to the server.

[0611] Step 8:

[0612] The server automatically generates and stores a prescription based on the selected treatment plan. As input, the selected treatment plan is entered into the server. As output, a prescription is generated and stored.

[0613] Step 9:

[0614] When the device returns to serving customers in a physical store, it reactivates the voice recognition system. The input is the customer's question or statement, which is entered into the microphone. The output is a recording of the speech and converted into text data.

[0615] Step 10:

[0616] The terminal sends the converted text data to the server as a prompt sentence for the generative AI model. As input, text data containing the customer's utterance is generated. As output, the generated prompt sentence is input to the server.

[0617] Step 11:

[0618] The server uses a generative AI model to generate a response text based on the prompt sentence. This process uses the OpenAI GPT-3 API. The prompt sentence is entered into the API as input. The response text is generated as output.

[0619] Step 12:

[0620] The server sends the generated answer text to the terminal, which outputs it as voice using the pyttsx3 library. As input, the answer text is input from the server. As output, the text is converted into voice so that the user (store clerk) can provide the answer to the customer.

[0621] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0622] The present invention is a system for streamlining the examination process for veterinarians and improving the accuracy of diagnoses, and by combining it with an emotion engine, the quality of examinations is further improved. Specific embodiments of the present invention are described below.

[0623] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or by using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and sends instructions to activate the device's voice recognition system and emotion engine.

[0624] Next, the user (veterinarian) examines the animal and verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. This text data is immediately sent to the server.

[0625] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0626] Furthermore, the emotion engine analyzes the content and voice patterns of the veterinarian's speech to recognize their emotions. For example, it can detect when the veterinarian is anxious or suspicious. The emotional information recognized by the emotion engine is added to the text data and recorded in the electronic medical record.

[0627] The server then automatically analyzes possible illnesses and treatment options based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal. The presented options are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The veterinarian selects the most appropriate option from the presented options and explains it to the owner.

[0628] For example, if the emotion engine detects a veterinarian's anxiety, it will adjust the priorities of potential diagnoses and treatment options to match that anxiety, allowing veterinarians to access the information they need more quickly and improving the accuracy of their diagnoses.

[0629] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0630] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[0631] In this way, the system of the present invention automates the recording of medical examination details and supports appropriate diagnosis and treatment based on emotional information, thereby improving the efficiency and accuracy of medical examinations.

[0632] The processing flow will be explained below.

[0633] Step 1:

[0634] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[0635] Step 2:

[0636] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[0637] Step 3:

[0638] The server receives the consultation start signal and sends an instruction to activate the voice recognition system and emotion engine to the terminal.

[0639] Step 4:

[0640] The device activates the voice recognition system and emotion engine, ready to record and transcribe the veterinarian's speech in real time.

[0641] Step 5:

[0642] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[0643] Step 6:

[0644] The device records the veterinarian's speech in real time, converts the audio data into text, and analyzes the emotions expressed in the speech using an emotion engine.

[0645] Step 7:

[0646] The device transmits the converted text data and analyzed emotional information to the server in real time.

[0647] Step 8:

[0648] The server analyzes the received text data and emotional information, and initializes and generates an electronic medical record of the consultation.

[0649] Step 9:

[0650] The server records keywords and medical examination details based on text data in the electronic medical record. At the same time, the emotion engine adds emotional information detected by the server to the medical record. Examples: "loss of appetite," "lack of energy," "doctor's concern."

[0651] Step 10:

[0652] The server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal.

[0653] Step 11:

[0654] The terminal receives the disease name and treatment options from the server and displays them to the veterinarian. The priority of the options may be changed based on the emotion information detected by the emotion engine.

[0655] Step 12:

[0656] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[0657] Step 13:

[0658] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[0659] Step 14:

[0660] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[0661] Step 15:

[0662] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[0663] In this way, by combining the emotion engine, it is possible to present an appropriate diagnosis and treatment plan that takes into account the examination information and the veterinarian's emotions, further improving the efficiency and accuracy of examinations.

[0664] Example 2

[0665] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0666] In conventional veterinary consultations, recording the details of the examination and improving the accuracy of the diagnosis have been issues. In particular, when veterinarians feel anxious or impatient, their emotions can affect the diagnosis. Furthermore, when the details of the examination are recorded manually, there is a high risk of errors or omissions. In addition, presenting possible diagnoses and selecting treatment plans is complicated, making it difficult to achieve an efficient examination process.

[0667] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for converting the veterinarian's remarks during the examination into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the veterinarian's remarks and voice patterns and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on emotion information. This enables the automatic recording of the examination details and the presentation of diagnosis candidates taking into account the veterinarian's emotion information.

[0668] "Speech recognition technology" is a technology that converts the voice spoken by a user into digital text in real time.

[0669] An "electronic medical record" is a database for digitally storing and managing the details of medical examinations, symptoms, diagnostic information, and treatment plans recorded during medical examinations.

[0670] The "means for automatically presenting disease names and treatment plan candidates" is a system that presents possible disease names and appropriate treatment plans to the user in list form based on the contents of the examination.

[0671] The "emotion engine" is a technology that analyzes the content and voice patterns of veterinarians' statements to recognize their emotions and generate emotional information.

[0672] "Emotional information" is data that represents the emotional state of the veterinarian and is used to adjust the priorities of candidate diagnoses.

[0673] The "means for adjusting the priority of candidate diagnoses" is an algorithm that changes the order of presented disease names and treatment plans, taking into account the veterinarian's emotional information.

[0674] "Means for converting into text in real time" refers to a function that instantly converts what the veterinarian says into text format and sends that text to the system.

[0675] An "operation indicating the start of examination" is a specific action taken by the veterinarian to start the examination, and includes, for example, pressing a button or inputting a voice command.

[0676] The present invention provides a system for streamlining the examination process by veterinarians and improving the accuracy of diagnoses. The system includes: means for converting speech from a veterinarian during an examination into text in real time using speech recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the content and speech patterns of the veterinarian's speech and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on the emotion information.

[0677] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or using a specific voice command. When the device detects this operation, it sends a notification that the examination has started to the server. The server then sends instructions to the device to start the voice recognition system and emotion engine.

[0678] While examining an animal, the user (veterinarian) verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. The converted text data is immediately sent to the server.

[0679] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded. Furthermore, the emotion engine analyzes the content of the veterinarian's speech and voice patterns to recognize the veterinarian's emotions. For example, it can detect when the veterinarian is anxious or suspicious. This emotional information is added to the text data and recorded in the electronic medical record.

[0680] Next, the server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents the candidates on the terminal. The presented candidates are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The user (veterinarian) selects the most appropriate one from the presented candidates and explains it to the owner. If the emotion engine detects the veterinarian's anxiety, the priority of the disease name and treatment plan candidates is adjusted to match that anxiety. This allows veterinarians to access the information they need quickly, improving the accuracy of diagnoses.

[0681] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0682] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[0683] An example prompt might be: "Please explain in detail, step by step, how your system would respond when the veterinarian says, 'This dog has no appetite and is listless.'"

[0684] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0685] Step 1:

[0686] The user (veterinarian) starts the examination on the terminal.

[0687] Input: Pressing the Start Visit button or a specific voice command.

[0688] Specific operation: The user presses the "Start consultation" button on the device's touchscreen or says the voice command "Start consultation."

[0689] Output: Consultation start event.

[0690] Step 2:

[0691] The terminal detects the start of a medical examination event and transmits a notice of the start of a medical examination to the server.

[0692] Input: Consultation start event.

[0693] Specific operation: The device sends an HTTP request to the server triggered by the start of a consultation event, which includes metadata such as the timestamp of the consultation start and the veterinarian's ID.

[0694] Output: Notification to the server that consultation has started.

[0695] Step 3:

[0696] The server receives notification that the consultation has started and sends instructions to the terminal to activate the voice recognition system and emotion engine.

[0697] Input: Notification of start of consultation from terminal.

[0698] Specific operation: As an HTTP response, the server sends a message to the terminal containing commands to start the voice recognition system and emotion engine.

[0699] Output: Instructions to the device to start the voice recognition system and emotion engine.

[0700] Step 4:

[0701] The user (veterinarian) examines the animal and verbally explains the animal's condition and observation results.

[0702] Input: Veterinarian statement.

[0703] Specific actions: The user (veterinarian) verbally explains the results of the examination, such as, "This dog has had no appetite recently and seems to be in a bad mood."

[0704] Output: The veterinarian's remarks are recorded as audio data.

[0705] Step 5:

[0706] The device records what the veterinarian says as audio data in real time and converts the audio data into text.

[0707] Input: Veterinarian speech data.

[0708] How it works: A speech recognition system converts speech data into text using machine learning models.

[0709] Output: Text data.

[0710] Step 6:

[0711] The terminal transmits the generated text data to the server.

[0712] Input: Text data.

[0713] Specific operation: The generated text data is packaged in JSON format and sent to the server as an HTTP request.

[0714] Output: Sends text data to the server.

[0715] Step 7:

[0716] The server generates an electronic medical record based on the received text data.

[0717] Input: Text data.

[0718] Specific operation: The server analyzes the received text data, identifies important medical details using a keyword extraction algorithm, and records them in the electronic medical record.

[0719] Output: Generated electronic medical record.

[0720] Step 8:

[0721] The server works in conjunction with the text data to run an emotion engine and analyze the veterinarian's emotions.

[0722] Input: Audio data.

[0723] How it works: The emotion engine uses machine learning models to analyze the tone, rate, emphasis, etc. of the voice and infer the veterinarian's emotional state.

[0724] Output: Emotional information.

[0725] Step 9:

[0726] Emotional information is added to the text data and recorded in the electronic medical record.

[0727] Input: Emotion information.

[0728] Specific operation: The veterinarian's emotion label is added to the text data and sent back to the server.

[0729] Output: Electronic medical record with emotional information added.

[0730] Step 10:

[0731] The server automatically analyzes the disease name and possible treatment options based on the contents of the electronic medical record and emotional information, and presents them on the terminal.

[0732] Input: Electronic medical record and emotional information.

[0733] How it works: The server analyzes the EMR and emotion information, uses machine learning models to search for relevant cases in the database, and sends a prioritized list of potential diagnoses and treatment options to the device.

[0734] Output: A list of potential diagnoses and treatment strategies.

[0735] Step 11:

[0736] The terminal displays the presented candidates in an interface that allows the user (veterinarian) to select one.

[0737] Input: A list of potential diagnoses and treatment strategies.

[0738] Specific operation: The terminal visually displays diagnostic candidates and treatment plans, and provides an interface that allows the user to easily make selections.

[0739] Output: User selection of potential diagnoses and treatment strategies.

[0740] Step 12:

[0741] The user (veterinarian) selects a candidate diagnosis, and the terminal automatically generates an appropriate prescription based on that.

[0742] Input: Selected candidate diagnoses.

[0743] Specific operation: The terminal generates a prescription based on the selected diagnostic candidate according to the type and size of the animal and stores it on the server.

[0744] Output: Auto-generated prescription.

[0745] (Application example 2)

[0746] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0747] The modern veterinarian consultation process requires accurate recording of consultation data, highly accurate diagnosis, and efficient treatment decisions. However, current methods require all of these processes to be done manually, which is not only time-consuming and labor-intensive, but also makes the accuracy of diagnosis highly dependent on the veterinarian's experience. Furthermore, as the demand for remote consultations increases, there is a need to provide consultations of the same quality remotely. To solve these challenges, a system that integrates speech recognition technology, emotion analysis technology, natural language processing technology, and remote consultation functionality is required.

[0748] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for converting the doctor's utterances during the consultation into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for analyzing the doctor's emotional state using emotion analysis technology and adding it to the text data; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record and emotion information; means for the doctor to select an appropriate disease name and treatment plan from the presented disease names and treatment plans; means for automatically generating a prescription based on the selected treatment plan; and means having a real-time video call function for remote consultations. This realizes efficient examination processes and improved diagnostic accuracy, making it possible to provide the same quality as conventional methods even in remote consultations.

[0749] "Speech recognition technology" is a technology that converts voice data into text data.

[0750] "Emotion analysis technology" is a technology that analyzes and determines a speaker's emotional state from voice and text data.

[0751] An "electronic medical record" is digital data used to manage the medical records of patients and animals on an information system, replacing paper medical records.

[0752] "Natural language processing technology" is a technology that allows computers to understand and process natural language.

[0753] "Remote consultation" is a method in which a doctor can conduct an examination via the Internet while being in a remote location from the patient or owner.

[0754] "Candidates for diagnosis and treatment" is a list of possible diagnoses and treatment methods based on the examination.

[0755] The "real-time video call function" is a function that allows two-way communication of video and audio in real time via the Internet.

[0756] A "prescription" is a document that lists specific medications or treatments prescribed by a physician for a patient or animal.

[0757] The present invention provides a system for improving the efficiency of the process when a veterinarian examines an animal and improving the accuracy of diagnosis. Specific embodiments are described below.

[0758] System configuration

[0759] 1. Use of voice recognition technology

[0760] The system uses speech recognition technology to convert what the veterinarian says into text in real time, using the Google Cloud Speech-to-Text API or a similar speech recognition API.

[0761] 2. Use of emotion analysis technology

[0762] Use IBM Watson Tone Analyzer or a similar sentiment analysis engine to analyze the veterinarian's speech and emotional state from their voice patterns.

[0763] 3. Generating Electronic Medical Records

[0764] Based on the converted text data and emotion information, an electronic medical record is automatically generated. The medical records are stored in a database system (e.g., MongoDB).

[0765] 4. Implementing real-time video calling

[0766] Incorporate video calling technology (e.g., WebRTC) to allow owners to receive consultations remotely.

[0767] 5. Diagnosis and treatment plan

[0768] Based on the generated electronic medical record and emotional information, symptoms are analyzed using natural language processing technology (e.g., generative AI model), and appropriate disease names and treatment options are generated and presented to the veterinarian.

[0769] 6. Automatic prescription generation

[0770] Based on the diagnostic results selected by the veterinarian, the appropriate prescription is automatically generated and provided to the owner in digital format.

[0771] Specific examples of functions

[0772] Process during consultation:

[0773] 1. Launching speech recognition and emotion analysis

[0774] When the veterinarian presses the "Start Examination" button, the system simultaneously activates voice recognition and emotion analysis technologies.

[0775] 2. Processing of veterinarians' speech data

[0776] When a veterinarian says, "This dog has been coughing a lot recently and seems lethargic," the voice recognition system converts this statement into text data in real time.

[0777] The emotion analysis engine detects emotions such as "worry" from the veterinarian's comments and adds them to the text data.

[0778] 3. Automatic generation of electronic medical records

[0779] Based on the converted text data and emotional information, content such as "Symptoms: cough, lethargy" and "Veterinarian's emotions: worried" are automatically recorded in the electronic medical record.

[0780] 4. Providing diagnosis and treatment guidelines

[0781] The system references electronic medical records and emotional information, and uses a "generative AI model" to suggest possible disease names such as "bronchitis" and "pneumonia," as well as treatment options.

[0782] The veterinarian will confirm the diagnosis based on the options presented.

[0783] 5. Automatic prescription generation

[0784] Based on the confirmed diagnosis, a specific prescription such as "antibiotics (300 mg) twice a day for 7 days" is automatically generated.

[0785] Example prompt sentence:

[0786] "Examination: This dog has been coughing a lot recently and seems lethargic.

[0787] Veterinarian's emotional state: Anxiety.

[0788] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[0789] In this way, the system supports veterinarians throughout the entire consultation process, enabling efficient and highly accurate consultations.

[0790] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0791] Step 1:

[0792] When the user (veterinarian) presses the button to start the consultation, the server simultaneously launches the speech recognition technology and emotion analysis technology. The input is the operation signal to start the consultation, and based on this, the server initializes the speech recognition API (e.g., Google Cloud Speech-to-Text) and emotion analysis engine (e.g., IBM Watson Tone Analyzer). The output is a state in which these technologies are ready to operate.

[0793] Step 2:

[0794] The device uses real-time video calling functionality to transmit video and audio between the owner and the veterinarian in two directions. The input is the audio and video data of the owner and the veterinarian, which is processed in real time through a video calling library (e.g., WebRTC). The output is a real-time video call session.

[0795] Step 3:

[0796] The device uses a voice recognition system to convert what the veterinarian says during the examination into text in real time. The input is the veterinarian's voice data, which is analyzed by the voice recognition API to generate text data. The output is the text data of the veterinarian's speech.

[0797] Step 4:

[0798] The server analyzes the text data using an emotion analysis engine to analyze the veterinarian's emotional state. The input is the text data of the veterinarian's comments, which the emotion analysis engine analyzes and adds emotional information. The output is text data with the added emotional information.

[0799] Step 5:

[0800] The server automatically generates an electronic medical record based on text data with added emotional information. The input is text data and emotional information, which are recorded in a database system (e.g., MongoDB) to generate the electronic medical record. The output is the electronic medical record data.

[0801] Step 6:

[0802] The server uses natural language processing technology to analyze symptoms using electronic medical record data and emotional information. The input is the electronic medical record data and emotional information, and by invoking the generative AI model with a prompt sentence, it generates appropriate disease names and treatment plan candidates. The output is disease names and treatment plan candidates.

[0803] Step 7:

[0804] The terminal presents the generated disease name and treatment plan candidate list to the veterinarian, allowing them to select the most appropriate diagnosis. The disease name and treatment plan candidate list are input, and the veterinarian makes the selection via the interface. The selected diagnosis result is output.

[0805] Step 8:

[0806] The server automatically generates an appropriate prescription based on the selected diagnostic results and provides it to the owner in digital form. The input is the selected diagnostic results, and the prescription data is generated based on this. The output is a digital prescription.

[0807] Specific actions

[0808] Example prompt sentence:

[0809] "Examination: This dog has been coughing a lot recently and seems lethargic.

[0810] Veterinarian's emotional state: Anxiety.

[0811] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[0812] Through this series of steps, the system streamlines the entire consultation process and enables highly accurate diagnosis and treatment.

[0813] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0814] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0815] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0816] [Third embodiment]

[0817] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0818] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0819] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0820] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0821] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0822] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0823] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0824] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0825] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0826] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0827] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0828] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0829] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function to automatically present the name of the disease and treatment plan. The operation of this system is described in detail below.

[0830] First, the user (veterinarian) starts the examination on the device. This can be done by pressing a button or using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and instructs it to start the device's voice recognition system.

[0831] Next, the user (veterinarian) comments while observing the animal's condition during the examination. For example, "This dog has lost its appetite recently and seems to be listless." The device records the veterinarian's comments in real time and converts the audio data into text. This converted text data is immediately sent to the server.

[0832] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0833] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while viewing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[0834] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0835] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information such as "lack of appetite" and "listlessness," and also presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[0836] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[0837] The processing flow will be explained below.

[0838] Step 1:

[0839] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[0840] Step 2:

[0841] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[0842] Step 3:

[0843] The server receives the consultation start signal and sends an instruction to start the voice recognition system to the terminal.

[0844] Step 4:

[0845] The device activates the voice recognition system and prepares to record and convert the veterinarian's speech into text in real time.

[0846] Step 5:

[0847] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[0848] Step 6:

[0849] The device records what the veterinarian says in real time and converts the audio data into text.

[0850] Step 7:

[0851] The terminal transmits the converted text data to the server in real time.

[0852] Step 8:

[0853] The server analyzes the received text data and initializes and generates the electronic medical record of the consultation.

[0854] Step 9:

[0855] The server records keywords and medical details based on the text data in the electronic medical record. Examples: "loss of appetite" and "lack of energy."

[0856] Step 10:

[0857] The server analyzes and generates possible disease names and treatment options based on the contents of the electronic medical record.

[0858] Step 11:

[0859] The server sends the generated disease name and treatment plan suggestions to the terminal and presents them to the user (veterinarian).

[0860] Step 12:

[0861] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[0862] Step 13:

[0863] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[0864] Step 14:

[0865] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[0866] Step 15:

[0867] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[0868] Example 1

[0869] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0870] In modern medical settings, doctors rely on manual input of information during consultations and the creation of prescriptions, resulting in problems such as long consultation times and a high risk of human error. Veterinarians, in particular, are required to accurately grasp the animal's condition, quickly record information, and provide an appropriate diagnosis and treatment plan. However, performing these tasks quickly and accurately is not easy, resulting in reduced consultation efficiency and problems with diagnostic accuracy. Given this background, there is a need for a system that can automate the consultation process and improve diagnostic accuracy.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0872] In this invention, the server includes a means for converting the doctor's speech during the consultation into text in real time using speech recognition technology, a means for automatically generating an electronic medical record based on the converted text, and a means for automatically presenting disease names and treatment options from the generated electronic medical record. This enables quick and accurate recording of the contents of the consultation, improving diagnostic accuracy and shortening consultation times.

[0873] "Speech recognition technology" is a technology that analyzes voice data and converts it into corresponding text data.

[0874] "Statements made by a doctor during an examination" refers to statements made by a doctor during an examination to verbalize the patient's condition, diagnosis, treatment plan, etc.

[0875] "Converting to text in real time" refers to instantly generating corresponding text data from voice input.

[0876] "Electronic medical records" refers to a system that electronically records and manages information such as medical examination details, patient conditions, diagnosis results, and treatment plans.

[0877] "Automatically generated" refers to a process in which the system automatically generates data using a dedicated algorithm or program, without the need for manual operation.

[0878] "Candidate disease names and treatment options" refers to disease names and treatment options that may be considered based on the patient's symptoms and examination results.

[0879] "Presenting" refers to the system showing the analysis results and generated information to the user.

[0880] The "operation indicating the start of a consultation" refers to an operation such as pressing a button or issuing a voice command performed by the user to notify the system that a consultation has started.

[0881] "Natural language processing technology" refers to the technology of analyzing natural language (human language) using a computer, and extracting and processing semantic information from text data.

[0882] "Symptom analysis" refers to the process of identifying specific symptoms from data entered during the consultation or recorded text, and then determining a diagnosis and treatment plan based on that information.

[0883] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function for automatically presenting the name of the disease and a treatment plan. An embodiment of this system is described in detail below.

[0884] First, the user (veterinarian) performs an operation on the device to start the examination. This can be done by pressing a specific button or by entering the voice command "Start examination." The device detects this operation and notifies the server that the examination has begun. The server receives this notification and instructs the device to start a voice recognition system. Examples of voice recognition systems that can be used include Google Speech-to-Text API and Nuance Dragon.

[0885] Next, the user (veterinarian) speaks while observing the animal's condition during the examination. For example, consider the case where the user says, "This dog has had no appetite recently and seems to be listless." The device records the veterinarian's speech in real time, sends the voice data to a speech recognition system, and converts it into text. This converted text data is immediately sent to the server.

[0886] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[0887] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while observing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[0888] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[0889] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information about "lack of appetite" and "listlessness," and then presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[0890] An example of a prompt is as follows:

[0891] Start the consultation

[0892] This dog has no appetite and is not energetic.

[0893] What is the possible name of the disease?

[0894] Generate a prescription.

[0895] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[0896] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0897] Step 1:

[0898] Notification of start of consultation

[0899] User:

[0900] To start the consultation, press a specific button on the device or enter the voice command "Start consultation."

[0901] Input: Press a button or voice command "Start consultation"

[0902] Output: Notification signal for start of consultation

[0903] Device:

[0904] The operation to start a medical examination is detected, and a request to notify the server of the start of the medical examination is sent.

[0905] Input: Consultation start notification signal

[0906] Output: Request to start consultation to the server

[0907] Step 2:

[0908] Activating voice recognition

[0909] server:

[0910] A request to start a consultation is received and an instruction is given to the terminal to start the voice recognition system.

[0911] Input: Consultation start request

[0912] Output: Voice recognition system activation command

[0913] Device:

[0914] The voice recognition system is started upon receiving instructions from the server.

[0915] Input: Voice recognition system activation command

[0916] Output: Start the voice recognition system

[0917] Step 3:

[0918] Audio recording and text conversion

[0919] User:

[0920] While observing the animal's condition, the examiner will comment on the examination results.

[0921] Input: Animal condition observations

[0922] Output: Statement of medical examination details

[0923] Device:

[0924] The user's speech is recorded in real time, and the speech data is sent to a speech recognition system and converted into text data.

[0925] Input: Speech about medical examination (audio data)

[0926] Output: Converted text data (diagnosis details)

[0927] Step 4:

[0928] Transmitting text data and generating electronic medical records

[0929] Device:

[0930] The converted text data is sent to the server.

[0931] Input: Converted text data

[0932] Output: Send text data to the server

[0933] server:

[0934] The received text data is analyzed and an electronic medical record is automatically generated.

[0935] Extract keywords and medical details from the audio text and add them to the electronic record.

[0936] Input: Received text data

[0937] Output: Generated electronic medical record

[0938] Step 5:

[0939] Presentation of disease name and treatment plan

[0940] server:

[0941] Analysis is performed based on electronic medical records to automatically generate possible disease names and treatment options.

[0942] The generated disease name and treatment plan candidates are sent to the terminal.

[0943] Input: Electronic Medical Record

[0944] Output: Disease name and treatment options

[0945] Device:

[0946] The received disease name and treatment plan suggestions are presented to the user on the interface.

[0947] Input: Disease name and treatment options

[0948] Output: Presented disease name and treatment options

[0949] Step 6:

[0950] Automatically generate and save prescriptions

[0951] User:

[0952] Select the appropriate disease name and treatment plan from the presented options.

[0953] Input: Disease name and treatment options

[0954] Output: Selected treatment strategy

[0955] Device:

[0956] A prescription is automatically generated based on the selected treatment plan and sent to the server.

[0957] Input: Selected treatment plan

[0958] Output: Generated prescription

[0959] server:

[0960] The generated prescription is stored in a database.

[0961] Input: Generated prescription

[0962] Output: Saved prescription

[0963] Through the above steps, this system records and automatically processes examination details in real time, improving diagnostic accuracy and examination efficiency.

[0964] (Application example 1)

[0965] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0966] In the traditional medical consultation process, doctors have to manually record the details of the consultation and select the diagnosis and treatment plan, which requires a great deal of time and effort. In addition, when dealing with customers in physical stores, it takes time and effort for store clerks to verbally answer customers' questions and provide product information. This can lead to a lack of efficiency and accuracy.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0968] In this invention, the server includes means for converting speech made by a doctor during an examination into text in real time using speech recognition technology, means for automatically generating an electronic medical record based on the converted text, means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record, means for the doctor to select an appropriate disease name and treatment plan from the presented ones, means for automatically generating a prescription based on the selected treatment plan, and means for converting conversations with customers in a physical store into text in real time using speech recognition technology and presenting product information and recommended products based on the conversation. This enables the efficiency and accuracy of the doctor's examination process, and enables more efficient customer service and improved customer satisfaction in physical stores.

[0969] "Speech recognition technology" is a technology that converts speech into text.

[0970] "Doctor's statements during consultation" refers to audio information uttered by the doctor during the consultation.

[0971] "Means for converting into text in real time" refers to devices or systems that have the ability to instantly convert speech into text using speech recognition technology.

[0972] An "electronic medical record" refers to a database that records and manages a patient's medical information in digital format.

[0973] "Means of automatic generation" refers to devices or software that have the ability to generate data or information without human intervention.

[0974] "Candidate disease names and treatment plans" refers to information that presents multiple possible disease names and treatment methods based on the contents of the examination.

[0975] "Means of presentation" refers to devices or software that have the function of displaying or providing information through a system, such as a screen or audio.

[0976] "Means for selecting the appropriate one" refers to devices or software that have an interface or function for selecting the most appropriate one from the presented candidates.

[0977] "Prescription" refers to a document that contains instructions for prescribing medication to a patient.

[0978] "Brick-and-mortar customers" refers to consumers who visit a physical sales location.

[0979] "Means of converting conversations into text in real time" refers to a speech recognition system that instantly converts conversations with customers into text.

[0980] "Means for displaying product information and recommended products" refers to devices or software that have the function of displaying related products or recommended products based on the customer's inquiry.

[0981] A "generative AI model" refers to a model for generating natural language text data using artificial intelligence technology.

[0982] A "prompt sentence" refers to the text data input to a generative AI model.

[0983]

[0984] The system of this invention converts doctor's speech during consultations into text in real time, and aims to not only automatically generate electronic medical records and prescriptions, but also to improve the efficiency of customer service in brick-and-mortar stores. This system is realized using speech recognition technology, natural language processing technology, and generative AI models.

[0985] The server first converts the doctor's speech during the consultation into text in real time using speech recognition technology. To do this, it records the speech using a microphone and converts the recorded speech data into text using the Python speech_recognition library. The converted text data is immediately sent to the server.

[0986] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are recorded in the electronic medical record. Next, natural language processing technology is used to generate disease names and treatment plan candidates based on the electronic medical record, and these are presented to the user, the doctor. The presentation is done through an interface, and the doctor selects the appropriate candidate. Based on this, the server automatically generates and saves a prescription.

[0987] Next, we will explain customer service in physical stores. The server converts conversations with customers in physical stores into text in real time using speech recognition technology. To do this, it also uses a microphone to record the voice and converts it into text using the speech_recognition library. The converted text is immediately used as a prompt for the generative AI model.

[0988] The generative AI model generates product information and recommendations based on a prompt provided in text format, using OpenAI's GPT-3 API and inputting the following prompt:

[0989] If a customer says 'Is this shampoo suitable for dry skin?' provide the following information:

[0990] The generated answer might be something like, "Yes, this shampoo is suitable for dry skin. It contains moisturizing ingredients and keeps the skin moist." The server uses the pyttsx3 library to output this generated text as speech again.

[0991] This system automatically records and analyzes medical examination details, and by streamlining customer service in physical stores, it is possible to reduce the burden on doctors and improve customer satisfaction.

[0992] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0993] Step 1:

[0994] The user (doctor or store clerk) starts the voice recognition system. As input, the user performs an operation (pressing a button or issuing a voice command) to start voice recognition. As output, the voice recognition system starts and goes into a voice recording standby state.

[0995] Step 2:

[0996] The user's (doctor or store clerk's) speech is recorded by the terminal's microphone. As input, the user's spoken voice data is input to the microphone. As output, the voice data is recorded and saved in binary format.

[0997] Step 3:

[0998] The device converts the recorded voice data into text data in real time. This process uses the speech_recognition library. The input is the recorded voice data. The output is the voice data converted into text data.

[0999] Step 4:

[1000] The converted text data is sent from the terminal to the server. As input, the text data is sent from the terminal. As output, the server receives the text data.

[1001] Step 5:

[1002] The server generates an electronic medical record based on the received text data. The text data is input to the server. The output is an electronic medical record that records the details of the examination and keywords.

[1003] Step 6:

[1004] The server generates disease names and treatment plan candidates based on the electronic medical records and presents them to the user (doctor). The input is the electronic medical record data entered into the server. The output is a list of disease names and treatment plan candidates that are displayed to the user.

[1005] Step 7:

[1006] The user (doctor) selects the appropriate disease name and treatment plan from the presented list. As input, a list of disease names and treatment plans is presented to the user. As output, the disease name and treatment plan selected by the user are sent to the server.

[1007] Step 8:

[1008] The server automatically generates and stores a prescription based on the selected treatment plan. As input, the selected treatment plan is entered into the server. As output, a prescription is generated and stored.

[1009] Step 9:

[1010] When the device returns to serving customers in a physical store, it reactivates the voice recognition system. The input is the customer's question or statement, which is entered into the microphone. The output is a recording of the speech and converted into text data.

[1011] Step 10:

[1012] The terminal sends the converted text data to the server as a prompt sentence for the generative AI model. As input, text data containing the customer's utterance is generated. As output, the generated prompt sentence is input to the server.

[1013] Step 11:

[1014] The server uses a generative AI model to generate a response text based on the prompt sentence. This process uses the OpenAI GPT-3 API. The prompt sentence is entered into the API as input. The response text is generated as output.

[1015] Step 12:

[1016] The server sends the generated answer text to the terminal, which outputs it as voice using the pyttsx3 library. As input, the answer text is input from the server. As output, the text is converted into voice so that the user (store clerk) can provide the answer to the customer.

[1017] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1018] The present invention is a system for streamlining the examination process for veterinarians and improving the accuracy of diagnoses, and by combining it with an emotion engine, the quality of examinations is further improved. Specific embodiments of the present invention are described below.

[1019] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or by using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and sends instructions to activate the device's voice recognition system and emotion engine.

[1020] Next, the user (veterinarian) examines the animal and verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. This text data is immediately sent to the server.

[1021] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[1022] Furthermore, the emotion engine analyzes the content and voice patterns of the veterinarian's speech to recognize their emotions. For example, it can detect when the veterinarian is anxious or suspicious. The emotional information recognized by the emotion engine is added to the text data and recorded in the electronic medical record.

[1023] The server then automatically analyzes possible illnesses and treatment options based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal. The presented options are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The veterinarian selects the most appropriate option from the presented options and explains it to the owner.

[1024] For example, if the emotion engine detects a veterinarian's anxiety, it will adjust the priorities of potential diagnoses and treatment options to match that anxiety, allowing veterinarians to access the information they need more quickly and improving the accuracy of their diagnoses.

[1025] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[1026] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[1027] In this way, the system of the present invention automates the recording of medical examination details and supports appropriate diagnosis and treatment based on emotional information, thereby improving the efficiency and accuracy of medical examinations.

[1028] The processing flow will be explained below.

[1029] Step 1:

[1030] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[1031] Step 2:

[1032] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[1033] Step 3:

[1034] The server receives the consultation start signal and sends an instruction to activate the voice recognition system and emotion engine to the terminal.

[1035] Step 4:

[1036] The device activates the voice recognition system and emotion engine, ready to record and transcribe the veterinarian's speech in real time.

[1037] Step 5:

[1038] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[1039] Step 6:

[1040] The device records the veterinarian's speech in real time, converts the audio data into text, and analyzes the emotions expressed in the speech using an emotion engine.

[1041] Step 7:

[1042] The device transmits the converted text data and analyzed emotional information to the server in real time.

[1043] Step 8:

[1044] The server analyzes the received text data and emotional information, and initializes and generates an electronic medical record of the consultation.

[1045] Step 9:

[1046] The server records keywords and medical examination details based on text data in the electronic medical record. At the same time, the emotion engine adds emotional information detected by the server to the medical record. Examples: "loss of appetite," "lack of energy," "doctor's concern."

[1047] Step 10:

[1048] The server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal.

[1049] Step 11:

[1050] The terminal receives the disease name and treatment options from the server and displays them to the veterinarian. The priority of the options may be changed based on the emotion information detected by the emotion engine.

[1051] Step 12:

[1052] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[1053] Step 13:

[1054] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[1055] Step 14:

[1056] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[1057] Step 15:

[1058] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[1059] In this way, by combining the emotion engine, it is possible to present an appropriate diagnosis and treatment plan that takes into account the examination information and the veterinarian's emotions, further improving the efficiency and accuracy of examinations.

[1060] Example 2

[1061] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1062] In conventional veterinary consultations, recording the details of the examination and improving the accuracy of the diagnosis have been issues. In particular, when veterinarians feel anxious or impatient, their emotions can affect the diagnosis. Furthermore, when the details of the examination are recorded manually, there is a high risk of errors or omissions. In addition, presenting possible diagnoses and selecting treatment plans is complicated, making it difficult to achieve an efficient examination process.

[1063] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for converting the veterinarian's remarks during the examination into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the veterinarian's remarks and voice patterns and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on emotion information. This enables the automatic recording of the examination details and the presentation of diagnosis candidates taking into account the veterinarian's emotion information.

[1064] "Speech recognition technology" is a technology that converts the voice spoken by a user into digital text in real time.

[1065] An "electronic medical record" is a database for digitally storing and managing the details of medical examinations, symptoms, diagnostic information, and treatment plans recorded during medical examinations.

[1066] The "means for automatically presenting disease names and treatment plan candidates" is a system that presents possible disease names and appropriate treatment plans to the user in list form based on the contents of the examination.

[1067] The "emotion engine" is a technology that analyzes the content and voice patterns of veterinarians' statements to recognize their emotions and generate emotional information.

[1068] "Emotional information" is data that represents the emotional state of the veterinarian and is used to adjust the priorities of candidate diagnoses.

[1069] The "means for adjusting the priority of candidate diagnoses" is an algorithm that changes the order of presented disease names and treatment plans, taking into account the veterinarian's emotional information.

[1070] "Means for converting into text in real time" refers to a function that instantly converts what the veterinarian says into text format and sends that text to the system.

[1071] An "operation indicating the start of examination" is a specific action taken by the veterinarian to start the examination, and includes, for example, pressing a button or inputting a voice command.

[1072] The present invention provides a system for streamlining the examination process by veterinarians and improving the accuracy of diagnoses. The system includes: means for converting speech from a veterinarian during an examination into text in real time using speech recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the content and speech patterns of the veterinarian's speech and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on the emotion information.

[1073] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or using a specific voice command. When the device detects this operation, it sends a notification that the examination has started to the server. The server then sends instructions to the device to start the voice recognition system and emotion engine.

[1074] While examining an animal, the user (veterinarian) verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. The converted text data is immediately sent to the server.

[1075] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded. Furthermore, the emotion engine analyzes the content of the veterinarian's speech and voice patterns to recognize the veterinarian's emotions. For example, it can detect when the veterinarian is anxious or suspicious. This emotional information is added to the text data and recorded in the electronic medical record.

[1076] Next, the server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents the candidates on the terminal. The presented candidates are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The user (veterinarian) selects the most appropriate one from the presented candidates and explains it to the owner. If the emotion engine detects the veterinarian's anxiety, the priority of the disease name and treatment plan candidates is adjusted to match that anxiety. This allows veterinarians to access the information they need quickly, improving the accuracy of diagnoses.

[1077] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[1078] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[1079] An example prompt might be: "Please explain in detail, step by step, how your system would respond when the veterinarian says, 'This dog has no appetite and is listless.'"

[1080] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1081] Step 1:

[1082] The user (veterinarian) starts the examination on the terminal.

[1083] Input: Pressing the Start Visit button or a specific voice command.

[1084] Specific operation: The user presses the "Start consultation" button on the device's touchscreen or says the voice command "Start consultation."

[1085] Output: Consultation start event.

[1086] Step 2:

[1087] The terminal detects the start of a medical examination event and transmits a notice of the start of a medical examination to the server.

[1088] Input: Consultation start event.

[1089] Specific operation: The device sends an HTTP request to the server triggered by the start of a consultation event, which includes metadata such as the timestamp of the consultation start and the veterinarian's ID.

[1090] Output: Notification to the server that consultation has started.

[1091] Step 3:

[1092] The server receives notification that the consultation has started and sends instructions to the terminal to activate the voice recognition system and emotion engine.

[1093] Input: Notification of start of consultation from terminal.

[1094] Specific operation: As an HTTP response, the server sends a message to the terminal containing commands to start the voice recognition system and emotion engine.

[1095] Output: Instructions to the device to start the voice recognition system and emotion engine.

[1096] Step 4:

[1097] The user (veterinarian) examines the animal and verbally explains the animal's condition and observation results.

[1098] Input: Veterinarian statement.

[1099] Specific actions: The user (veterinarian) verbally explains the results of the examination, such as, "This dog has had no appetite recently and seems to be in a bad mood."

[1100] Output: The veterinarian's remarks are recorded as audio data.

[1101] Step 5:

[1102] The device records what the veterinarian says as audio data in real time and converts the audio data into text.

[1103] Input: Veterinarian speech data.

[1104] How it works: A speech recognition system converts speech data into text using machine learning models.

[1105] Output: Text data.

[1106] Step 6:

[1107] The terminal transmits the generated text data to the server.

[1108] Input: Text data.

[1109] Specific operation: The generated text data is packaged in JSON format and sent to the server as an HTTP request.

[1110] Output: Sends text data to the server.

[1111] Step 7:

[1112] The server generates an electronic medical record based on the received text data.

[1113] Input: Text data.

[1114] Specific operation: The server analyzes the received text data, identifies important medical details using a keyword extraction algorithm, and records them in the electronic medical record.

[1115] Output: Generated electronic medical record.

[1116] Step 8:

[1117] The server works in conjunction with the text data to run an emotion engine and analyze the veterinarian's emotions.

[1118] Input: Audio data.

[1119] How it works: The emotion engine uses machine learning models to analyze the tone, rate, emphasis, etc. of the voice and infer the veterinarian's emotional state.

[1120] Output: Emotional information.

[1121] Step 9:

[1122] Emotional information is added to the text data and recorded in the electronic medical record.

[1123] Input: Emotion information.

[1124] Specific operation: The veterinarian's emotion label is added to the text data and sent back to the server.

[1125] Output: Electronic medical record with emotional information added.

[1126] Step 10:

[1127] The server automatically analyzes the disease name and possible treatment options based on the contents of the electronic medical record and emotional information, and presents them on the terminal.

[1128] Input: Electronic medical record and emotional information.

[1129] How it works: The server analyzes the EMR and emotion information, uses machine learning models to search for relevant cases in the database, and sends a prioritized list of potential diagnoses and treatment options to the device.

[1130] Output: A list of potential diagnoses and treatment strategies.

[1131] Step 11:

[1132] The terminal displays the presented candidates in an interface that allows the user (veterinarian) to select one.

[1133] Input: A list of potential diagnoses and treatment strategies.

[1134] Specific operation: The terminal visually displays diagnostic candidates and treatment plans, and provides an interface that allows the user to easily make selections.

[1135] Output: User selection of potential diagnoses and treatment strategies.

[1136] Step 12:

[1137] The user (veterinarian) selects a candidate diagnosis, and the terminal automatically generates an appropriate prescription based on that.

[1138] Input: Selected candidate diagnoses.

[1139] Specific operation: The terminal generates a prescription based on the selected diagnostic candidate according to the type and size of the animal and stores it on the server.

[1140] Output: Auto-generated prescription.

[1141] (Application example 2)

[1142] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1143] The modern veterinarian consultation process requires accurate recording of consultation data, highly accurate diagnosis, and efficient treatment decisions. However, current methods require all of these processes to be done manually, which is not only time-consuming and labor-intensive, but also makes the accuracy of diagnosis highly dependent on the veterinarian's experience. Furthermore, as the demand for remote consultations increases, there is a need to provide consultations of the same quality remotely. To solve these challenges, a system that integrates speech recognition technology, emotion analysis technology, natural language processing technology, and remote consultation functionality is required.

[1144] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for converting the doctor's utterances during the consultation into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for analyzing the doctor's emotional state using emotion analysis technology and adding it to the text data; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record and emotion information; means for the doctor to select an appropriate disease name and treatment plan from the presented disease names and treatment plans; means for automatically generating a prescription based on the selected treatment plan; and means having a real-time video call function for remote consultations. This realizes efficient examination processes and improved diagnostic accuracy, making it possible to provide the same quality as conventional methods even in remote consultations.

[1145] "Speech recognition technology" is a technology that converts voice data into text data.

[1146] "Emotion analysis technology" is a technology that analyzes and determines a speaker's emotional state from voice and text data.

[1147] An "electronic medical record" is digital data used to manage the medical records of patients and animals on an information system, replacing paper medical records.

[1148] "Natural language processing technology" is a technology that allows computers to understand and process natural language.

[1149] "Remote consultation" is a method in which a doctor can conduct an examination via the Internet while being in a remote location from the patient or owner.

[1150] "Candidates for diagnosis and treatment" is a list of possible diagnoses and treatment methods based on the examination.

[1151] The "real-time video call function" is a function that allows two-way communication of video and audio in real time via the Internet.

[1152] A "prescription" is a document that lists specific medications or treatments prescribed by a physician for a patient or animal.

[1153] The present invention provides a system for improving the efficiency of the process when a veterinarian examines an animal and improving the accuracy of diagnosis. Specific embodiments are described below.

[1154] System configuration

[1155] 1. Use of voice recognition technology

[1156] The system uses speech recognition technology to convert what the veterinarian says into text in real time, using the Google Cloud Speech-to-Text API or a similar speech recognition API.

[1157] 2. Use of emotion analysis technology

[1158] Use IBM Watson Tone Analyzer or a similar sentiment analysis engine to analyze the veterinarian's speech and emotional state from their voice patterns.

[1159] 3. Generating Electronic Medical Records

[1160] Based on the converted text data and emotion information, an electronic medical record is automatically generated. The medical records are stored in a database system (e.g., MongoDB).

[1161] 4. Implementing real-time video calling

[1162] Incorporate video calling technology (e.g., WebRTC) to allow owners to receive consultations remotely.

[1163] 5. Diagnosis and treatment plan

[1164] Based on the generated electronic medical record and emotional information, symptoms are analyzed using natural language processing technology (e.g., generative AI model), and appropriate disease names and treatment options are generated and presented to the veterinarian.

[1165] 6. Automatic prescription generation

[1166] Based on the diagnostic results selected by the veterinarian, the appropriate prescription is automatically generated and provided to the owner in digital format.

[1167] Specific examples of functions

[1168] Process during consultation:

[1169] 1. Launching speech recognition and emotion analysis

[1170] When the veterinarian presses the "Start Examination" button, the system simultaneously activates voice recognition and emotion analysis technologies.

[1171] 2. Processing of veterinarians' speech data

[1172] When a veterinarian says, "This dog has been coughing a lot recently and seems lethargic," the voice recognition system converts this statement into text data in real time.

[1173] The emotion analysis engine detects emotions such as "worry" from the veterinarian's comments and adds them to the text data.

[1174] 3. Automatic generation of electronic medical records

[1175] Based on the converted text data and emotional information, content such as "Symptoms: cough, lethargy" and "Veterinarian's emotions: worried" are automatically recorded in the electronic medical record.

[1176] 4. Providing diagnosis and treatment guidelines

[1177] The system references electronic medical records and emotional information, and uses a "generative AI model" to suggest possible disease names such as "bronchitis" and "pneumonia," as well as treatment options.

[1178] The veterinarian will confirm the diagnosis based on the options presented.

[1179] 5. Automatic prescription generation

[1180] Based on the confirmed diagnosis, a specific prescription such as "antibiotics (300 mg) twice a day for 7 days" is automatically generated.

[1181] Example prompt sentence:

[1182] "Examination: This dog has been coughing a lot recently and seems lethargic.

[1183] Veterinarian's emotional state: Anxiety.

[1184] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[1185] In this way, the system supports veterinarians throughout the entire consultation process, enabling efficient and highly accurate consultations.

[1186] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1187] Step 1:

[1188] When the user (veterinarian) presses the button to start the consultation, the server simultaneously launches the speech recognition technology and emotion analysis technology. The input is the operation signal to start the consultation, and based on this, the server initializes the speech recognition API (e.g., Google Cloud Speech-to-Text) and emotion analysis engine (e.g., IBM Watson Tone Analyzer). The output is a state in which these technologies are ready to operate.

[1189] Step 2:

[1190] The device uses real-time video calling functionality to transmit video and audio between the owner and the veterinarian in two directions. The input is the audio and video data of the owner and the veterinarian, which is processed in real time through a video calling library (e.g., WebRTC). The output is a real-time video call session.

[1191] Step 3:

[1192] The device uses a voice recognition system to convert what the veterinarian says during the examination into text in real time. The input is the veterinarian's voice data, which is analyzed by the voice recognition API to generate text data. The output is the text data of the veterinarian's speech.

[1193] Step 4:

[1194] The server analyzes the text data using an emotion analysis engine to analyze the veterinarian's emotional state. The input is the text data of the veterinarian's comments, which the emotion analysis engine analyzes and adds emotional information. The output is text data with the added emotional information.

[1195] Step 5:

[1196] The server automatically generates an electronic medical record based on text data with added emotional information. The input is text data and emotional information, which are recorded in a database system (e.g., MongoDB) to generate the electronic medical record. The output is the electronic medical record data.

[1197] Step 6:

[1198] The server uses natural language processing technology to analyze symptoms using electronic medical record data and emotional information. The input is the electronic medical record data and emotional information, and by invoking the generative AI model with a prompt sentence, it generates appropriate disease names and treatment plan candidates. The output is disease names and treatment plan candidates.

[1199] Step 7:

[1200] The terminal presents the generated disease name and treatment plan candidate list to the veterinarian, allowing them to select the most appropriate diagnosis. The disease name and treatment plan candidate list are input, and the veterinarian makes the selection via the interface. The selected diagnosis result is output.

[1201] Step 8:

[1202] The server automatically generates an appropriate prescription based on the selected diagnostic results and provides it to the owner in digital form. The input is the selected diagnostic results, and the prescription data is generated based on this. The output is a digital prescription.

[1203] Specific actions

[1204] Example prompt sentence:

[1205] "Examination: This dog has been coughing a lot recently and seems lethargic.

[1206] Veterinarian's emotional state: Anxiety.

[1207] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[1208] Through this series of steps, the system streamlines the entire consultation process and enables highly accurate diagnosis and treatment.

[1209] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1210] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1211] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1212] [Fourth embodiment]

[1213] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1214] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1215] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1216] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1217] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1218] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1219] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1220] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1221] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1222] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1223] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1224] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1225] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1226] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function to automatically present the name of the disease and treatment plan. The operation of this system is described in detail below.

[1227] First, the user (veterinarian) starts the examination on the device. This can be done by pressing a button or using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and instructs it to start the device's voice recognition system.

[1228] Next, the user (veterinarian) comments while observing the animal's condition during the examination. For example, "This dog has lost its appetite recently and seems to be listless." The device records the veterinarian's comments in real time and converts the audio data into text. This converted text data is immediately sent to the server.

[1229] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[1230] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while viewing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[1231] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[1232] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information such as "lack of appetite" and "listlessness," and also presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[1233] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[1234] The processing flow will be explained below.

[1235] Step 1:

[1236] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[1237] Step 2:

[1238] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[1239] Step 3:

[1240] The server receives the consultation start signal and sends an instruction to start the voice recognition system to the terminal.

[1241] Step 4:

[1242] The device activates the voice recognition system and prepares to record and convert the veterinarian's speech into text in real time.

[1243] Step 5:

[1244] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[1245] Step 6:

[1246] The device records what the veterinarian says in real time and converts the audio data into text.

[1247] Step 7:

[1248] The terminal transmits the converted text data to the server in real time.

[1249] Step 8:

[1250] The server analyzes the received text data and initializes and generates the electronic medical record of the consultation.

[1251] Step 9:

[1252] The server records keywords and medical details based on the text data in the electronic medical record. Examples: "loss of appetite" and "lack of energy."

[1253] Step 10:

[1254] The server analyzes and generates possible disease names and treatment options based on the contents of the electronic medical record.

[1255] Step 11:

[1256] The server sends the generated disease name and treatment plan suggestions to the terminal and presents them to the user (veterinarian).

[1257] Step 12:

[1258] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[1259] Step 13:

[1260] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[1261] Step 14:

[1262] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[1263] Step 15:

[1264] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[1265] Example 1

[1266] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1267] In modern medical settings, doctors rely on manual input of information during consultations and the creation of prescriptions, resulting in problems such as long consultation times and a high risk of human error. Veterinarians, in particular, are required to accurately grasp the animal's condition, quickly record information, and provide an appropriate diagnosis and treatment plan. However, performing these tasks quickly and accurately is not easy, resulting in reduced consultation efficiency and problems with diagnostic accuracy. Given this background, there is a need for a system that can automate the consultation process and improve diagnostic accuracy.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1269] In this invention, the server includes a means for converting the doctor's speech during the consultation into text in real time using speech recognition technology, a means for automatically generating an electronic medical record based on the converted text, and a means for automatically presenting disease names and treatment options from the generated electronic medical record. This enables quick and accurate recording of the contents of the consultation, improving diagnostic accuracy and shortening consultation times.

[1270] "Speech recognition technology" is a technology that analyzes voice data and converts it into corresponding text data.

[1271] "Statements made by a doctor during an examination" refers to statements made by a doctor during an examination to verbalize the patient's condition, diagnosis, treatment plan, etc.

[1272] "Converting to text in real time" refers to instantly generating corresponding text data from voice input.

[1273] "Electronic medical records" refers to a system that electronically records and manages information such as medical examination details, patient conditions, diagnosis results, and treatment plans.

[1274] "Automatically generated" refers to a process in which the system automatically generates data using a dedicated algorithm or program, without the need for manual operation.

[1275] "Candidate disease names and treatment options" refers to disease names and treatment options that may be considered based on the patient's symptoms and examination results.

[1276] "Presenting" refers to the system showing the analysis results and generated information to the user.

[1277] The "operation indicating the start of a consultation" refers to an operation such as pressing a button or issuing a voice command performed by the user to notify the system that a consultation has started.

[1278] "Natural language processing technology" refers to the technology of analyzing natural language (human language) using a computer, and extracting and processing semantic information from text data.

[1279] "Symptom analysis" refers to the process of identifying specific symptoms from data entered during the consultation or recorded text, and then determining a diagnosis and treatment plan based on that information.

[1280] The present invention is a system for streamlining the veterinarian's consultation process and improving the accuracy of diagnoses. This system uses voice recognition technology to convert the veterinarian's consultation content into text in real time, and automatically generates electronic medical records and prescriptions based on that text. It also includes a function for automatically presenting the name of the disease and a treatment plan. An embodiment of this system is described in detail below.

[1281] First, the user (veterinarian) performs an operation on the device to start the examination. This can be done by pressing a specific button or by entering the voice command "Start examination." The device detects this operation and notifies the server that the examination has begun. The server receives this notification and instructs the device to start a voice recognition system. Examples of voice recognition systems that can be used include Google Speech-to-Text API and Nuance Dragon.

[1282] Next, the user (veterinarian) speaks while observing the animal's condition during the examination. For example, consider the case where the user says, "This dog has had no appetite recently and seems to be listless." The device records the veterinarian's speech in real time, sends the voice data to a speech recognition system, and converts it into text. This converted text data is immediately sent to the server.

[1283] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[1284] Next, the server automatically analyzes possible illnesses and treatment options based on the electronic medical records, and presents the options on the terminal. An interface is provided so that the user (veterinarian) can select the option while observing the animal's condition. The veterinarian selects the most appropriate option from the options presented and explains it to the owner.

[1285] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[1286] As a concrete example, consider the case where a veterinarian is examining a dog. The veterinarian begins the examination, and the terminal activates the voice recognition system. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. The server generates an electronic medical record based on the information about "lack of appetite" and "listlessness," and then presents possible illnesses such as "gastrointestinal upset" and "infection" along with treatment options. When the veterinarian selects "gastrointestinal upset," an appropriate prescription is automatically generated based on this information.

[1287] An example of a prompt is as follows:

[1288] Start the consultation

[1289] This dog has no appetite and is not energetic.

[1290] What is the possible name of the disease?

[1291] Generate a prescription.

[1292] In this way, the system of the present invention automates the recording of medical examination details and supports the selection of disease names and treatment plans, thereby improving the efficiency and accuracy of medical examinations.

[1293] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1294] Step 1:

[1295] Notification of start of consultation

[1296] User:

[1297] To start the consultation, press a specific button on the device or enter the voice command "Start consultation."

[1298] Input: Press a button or voice command "Start consultation"

[1299] Output: Notification signal for start of consultation

[1300] Device:

[1301] The operation to start a medical examination is detected, and a request to notify the server of the start of the medical examination is sent.

[1302] Input: Consultation start notification signal

[1303] Output: Request to start consultation to the server

[1304] Step 2:

[1305] Activating voice recognition

[1306] server:

[1307] A request to start a consultation is received and an instruction is given to the terminal to start the voice recognition system.

[1308] Input: Consultation start request

[1309] Output: Voice recognition system activation command

[1310] Device:

[1311] The voice recognition system is started upon receiving instructions from the server.

[1312] Input: Voice recognition system activation command

[1313] Output: Start the voice recognition system

[1314] Step 3:

[1315] Audio recording and text conversion

[1316] User:

[1317] While observing the animal's condition, the examiner will comment on the examination results.

[1318] Input: Animal condition observations

[1319] Output: Statement of medical examination details

[1320] Device:

[1321] The user's speech is recorded in real time, and the speech data is sent to a speech recognition system and converted into text data.

[1322] Input: Speech about medical examination (audio data)

[1323] Output: Converted text data (diagnosis details)

[1324] Step 4:

[1325] Transmitting text data and generating electronic medical records

[1326] Device:

[1327] The converted text data is sent to the server.

[1328] Input: Converted text data

[1329] Output: Send text data to the server

[1330] server:

[1331] The received text data is analyzed and an electronic medical record is automatically generated.

[1332] Extract keywords and medical details from the audio text and add them to the electronic record.

[1333] Input: Received text data

[1334] Output: Generated electronic medical record

[1335] Step 5:

[1336] Presentation of disease name and treatment plan

[1337] server:

[1338] Analysis is performed based on electronic medical records to automatically generate possible disease names and treatment options.

[1339] The generated disease name and treatment plan candidates are sent to the terminal.

[1340] Input: Electronic Medical Record

[1341] Output: Disease name and treatment options

[1342] Device:

[1343] The received disease name and treatment plan suggestions are presented to the user on the interface.

[1344] Input: Disease name and treatment options

[1345] Output: Presented disease name and treatment options

[1346] Step 6:

[1347] Automatically generate and save prescriptions

[1348] User:

[1349] Select the appropriate disease name and treatment plan from the presented options.

[1350] Input: Disease name and treatment options

[1351] Output: Selected treatment strategy

[1352] Device:

[1353] A prescription is automatically generated based on the selected treatment plan and sent to the server.

[1354] Input: Selected treatment plan

[1355] Output: Generated prescription

[1356] server:

[1357] The generated prescription is stored in a database.

[1358] Input: Generated prescription

[1359] Output: Saved prescription

[1360] Through the above steps, this system records and automatically processes examination details in real time, improving diagnostic accuracy and examination efficiency.

[1361] (Application example 1)

[1362] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1363] In the traditional medical consultation process, doctors have to manually record the details of the consultation and select the diagnosis and treatment plan, which requires a great deal of time and effort. In addition, when dealing with customers in physical stores, it takes time and effort for store clerks to verbally answer customers' questions and provide product information. This can lead to a lack of efficiency and accuracy.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1365] In this invention, the server includes means for converting speech made by a doctor during an examination into text in real time using speech recognition technology, means for automatically generating an electronic medical record based on the converted text, means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record, means for the doctor to select an appropriate disease name and treatment plan from the presented ones, means for automatically generating a prescription based on the selected treatment plan, and means for converting conversations with customers in a physical store into text in real time using speech recognition technology and presenting product information and recommended products based on the conversation. This enables the efficiency and accuracy of the doctor's examination process, and enables more efficient customer service and improved customer satisfaction in physical stores.

[1366] "Speech recognition technology" is a technology that converts speech into text.

[1367] "Doctor's statements during consultation" refers to audio information uttered by the doctor during the consultation.

[1368] "Means for converting into text in real time" refers to devices or systems that have the ability to instantly convert speech into text using speech recognition technology.

[1369] An "electronic medical record" refers to a database that records and manages a patient's medical information in digital format.

[1370] "Means of automatic generation" refers to devices or software that have the ability to generate data or information without human intervention.

[1371] "Candidate disease names and treatment plans" refers to information that presents multiple possible disease names and treatment methods based on the contents of the examination.

[1372] "Means of presentation" refers to devices or software that have the function of displaying or providing information through a system, such as a screen or audio.

[1373] "Means for selecting the appropriate one" refers to devices or software that have an interface or function for selecting the most appropriate one from the presented candidates.

[1374] "Prescription" refers to a document that contains instructions for prescribing medication to a patient.

[1375] "Brick-and-mortar customers" refers to consumers who visit a physical sales location.

[1376] "Means of converting conversations into text in real time" refers to a speech recognition system that instantly converts conversations with customers into text.

[1377] "Means for displaying product information and recommended products" refers to devices or software that have the function of displaying related products or recommended products based on the customer's inquiry.

[1378] A "generative AI model" refers to a model for generating natural language text data using artificial intelligence technology.

[1379] A "prompt sentence" refers to the text data input to a generative AI model.

[1380]

[1381] The system of this invention converts doctor's speech during consultations into text in real time, and aims to not only automatically generate electronic medical records and prescriptions, but also to improve the efficiency of customer service in brick-and-mortar stores. This system is realized using speech recognition technology, natural language processing technology, and generative AI models.

[1382] The server first converts the doctor's speech during the consultation into text in real time using speech recognition technology. To do this, it records the speech using a microphone and converts the recorded speech data into text using the Python speech_recognition library. The converted text data is immediately sent to the server.

[1383] The server automatically generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are recorded in the electronic medical record. Next, natural language processing technology is used to generate disease names and treatment plan candidates based on the electronic medical record, and these are presented to the user, the doctor. The presentation is done through an interface, and the doctor selects the appropriate candidate. Based on this, the server automatically generates and saves a prescription.

[1384] Next, we will explain customer service in physical stores. The server converts conversations with customers in physical stores into text in real time using speech recognition technology. To do this, it also uses a microphone to record the voice and converts it into text using the speech_recognition library. The converted text is immediately used as a prompt for the generative AI model.

[1385] The generative AI model generates product information and recommendations based on a prompt provided in text format, using OpenAI's GPT-3 API and inputting the following prompt:

[1386] If a customer says 'Is this shampoo suitable for dry skin?' provide the following information:

[1387] The generated answer might be something like, "Yes, this shampoo is suitable for dry skin. It contains moisturizing ingredients and keeps the skin moist." The server uses the pyttsx3 library to output this generated text as speech again.

[1388] This system automatically records and analyzes medical examination details, and by streamlining customer service in physical stores, it is possible to reduce the burden on doctors and improve customer satisfaction.

[1389] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1390] Step 1:

[1391] The user (doctor or store clerk) starts the voice recognition system. As input, the user performs an operation (pressing a button or issuing a voice command) to start voice recognition. As output, the voice recognition system starts and goes into a voice recording standby state.

[1392] Step 2:

[1393] The user's (doctor or store clerk's) speech is recorded by the terminal's microphone. As input, the user's spoken voice data is input to the microphone. As output, the voice data is recorded and saved in binary format.

[1394] Step 3:

[1395] The device converts the recorded voice data into text data in real time. This process uses the speech_recognition library. The input is the recorded voice data. The output is the voice data converted into text data.

[1396] Step 4:

[1397] The converted text data is sent from the terminal to the server. As input, the text data is sent from the terminal. As output, the server receives the text data.

[1398] Step 5:

[1399] The server generates an electronic medical record based on the received text data. The text data is input to the server. The output is an electronic medical record that records the details of the examination and keywords.

[1400] Step 6:

[1401] The server generates disease names and treatment plan candidates based on the electronic medical records and presents them to the user (doctor). The input is the electronic medical record data entered into the server. The output is a list of disease names and treatment plan candidates that are displayed to the user.

[1402] Step 7:

[1403] The user (doctor) selects the appropriate disease name and treatment plan from the presented list. As input, a list of disease names and treatment plans is presented to the user. As output, the disease name and treatment plan selected by the user are sent to the server.

[1404] Step 8:

[1405] The server automatically generates and stores a prescription based on the selected treatment plan. As input, the selected treatment plan is entered into the server. As output, a prescription is generated and stored.

[1406] Step 9:

[1407] When the device returns to serving customers in a physical store, it reactivates the voice recognition system. The input is the customer's question or statement, which is entered into the microphone. The output is a recording of the speech and converted into text data.

[1408] Step 10:

[1409] The terminal sends the converted text data to the server as a prompt sentence for the generative AI model. As input, text data containing the customer's utterance is generated. As output, the generated prompt sentence is input to the server.

[1410] Step 11:

[1411] The server uses a generative AI model to generate a response text based on the prompt sentence. This process uses the OpenAI GPT-3 API. The prompt sentence is entered into the API as input. The response text is generated as output.

[1412] Step 12:

[1413] The server sends the generated answer text to the terminal, which outputs it as voice using the pyttsx3 library. As input, the answer text is input from the server. As output, the text is converted into voice so that the user (store clerk) can provide the answer to the customer.

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] The present invention is a system for streamlining the examination process for veterinarians and improving the accuracy of diagnoses, and by combining it with an emotion engine, the quality of examinations is further improved. Specific embodiments of the present invention are described below.

[1416] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or by using a specific voice command. The device detects this operation and notifies the server that the examination has started. The server receives this notification and sends instructions to activate the device's voice recognition system and emotion engine.

[1417] Next, the user (veterinarian) examines the animal and verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. This text data is immediately sent to the server.

[1418] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded.

[1419] Furthermore, the emotion engine analyzes the content and voice patterns of the veterinarian's speech to recognize their emotions. For example, it can detect when the veterinarian is anxious or suspicious. The emotional information recognized by the emotion engine is added to the text data and recorded in the electronic medical record.

[1420] The server then automatically analyzes possible illnesses and treatment options based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal. The presented options are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The veterinarian selects the most appropriate option from the presented options and explains it to the owner.

[1421] For example, if the emotion engine detects a veterinarian's anxiety, it will adjust the priorities of potential diagnoses and treatment options to match that anxiety, allowing veterinarians to access the information they need more quickly and improving the accuracy of their diagnoses.

[1422] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[1423] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[1424] In this way, the system of the present invention automates the recording of medical examination details and supports appropriate diagnosis and treatment based on emotional information, thereby improving the efficiency and accuracy of medical examinations.

[1425] The processing flow will be explained below.

[1426] Step 1:

[1427] The user (veterinarian) starts the examination on the terminal by pressing the examination start button or using a voice command.

[1428] Step 2:

[1429] The terminal detects the operation to start the examination and transmits a signal to start the examination to the server.

[1430] Step 3:

[1431] The server receives the consultation start signal and sends an instruction to activate the voice recognition system and emotion engine to the terminal.

[1432] Step 4:

[1433] The device activates the voice recognition system and emotion engine, ready to record and transcribe the veterinarian's speech in real time.

[1434] Step 5:

[1435] The user (veterinarian) verbally explains the animal's condition while observing it during the examination. Example: "This dog has had a poor appetite recently and seems to be listless."

[1436] Step 6:

[1437] The device records the veterinarian's speech in real time, converts the audio data into text, and analyzes the emotions expressed in the speech using an emotion engine.

[1438] Step 7:

[1439] The device transmits the converted text data and analyzed emotional information to the server in real time.

[1440] Step 8:

[1441] The server analyzes the received text data and emotional information, and initializes and generates an electronic medical record of the consultation.

[1442] Step 9:

[1443] The server records keywords and medical examination details based on text data in the electronic medical record. At the same time, the emotion engine adds emotional information detected by the server to the medical record. Examples: "loss of appetite," "lack of energy," "doctor's concern."

[1444] Step 10:

[1445] The server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents candidates on the terminal.

[1446] Step 11:

[1447] The terminal receives the disease name and treatment options from the server and displays them to the veterinarian. The priority of the options may be changed based on the emotion information detected by the emotion engine.

[1448] Step 12:

[1449] The user (veterinarian) selects the appropriate disease name and treatment plan from the presented options. Example: "Gastrointestinal problems"

[1450] Step 13:

[1451] The user (veterinarian) inputs the selection results into the terminal and sends the results to the server.

[1452] Step 14:

[1453] Based on the selected treatment plan, the server automatically generates a prescription, taking into account the animal's basic information (species, size, etc.).

[1454] Step 15:

[1455] The server adds the generated prescription to the electronic medical record and stores it as part of the overall medical record.

[1456] In this way, by combining the emotion engine, it is possible to present an appropriate diagnosis and treatment plan that takes into account the examination information and the veterinarian's emotions, further improving the efficiency and accuracy of examinations.

[1457] Example 2

[1458] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] In conventional veterinary consultations, recording the details of the examination and improving the accuracy of the diagnosis have been issues. In particular, when veterinarians feel anxious or impatient, their emotions can affect the diagnosis. Furthermore, when the details of the examination are recorded manually, there is a high risk of errors or omissions. In addition, presenting possible diagnoses and selecting treatment plans is complicated, making it difficult to achieve an efficient examination process.

[1460] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for converting the veterinarian's remarks during the examination into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the veterinarian's remarks and voice patterns and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on emotion information. This enables the automatic recording of the examination details and the presentation of diagnosis candidates taking into account the veterinarian's emotion information.

[1461] "Speech recognition technology" is a technology that converts the voice spoken by a user into digital text in real time.

[1462] An "electronic medical record" is a database for digitally storing and managing the details of medical examinations, symptoms, diagnostic information, and treatment plans recorded during medical examinations.

[1463] The "means for automatically presenting disease names and treatment plan candidates" is a system that presents possible disease names and appropriate treatment plans to the user in list form based on the contents of the examination.

[1464] The "emotion engine" is a technology that analyzes the content and voice patterns of veterinarians' statements to recognize their emotions and generate emotional information.

[1465] "Emotional information" is data that represents the emotional state of the veterinarian and is used to adjust the priorities of candidate diagnoses.

[1466] The "means for adjusting the priority of candidate diagnoses" is an algorithm that changes the order of presented disease names and treatment plans, taking into account the veterinarian's emotional information.

[1467] "Means for converting into text in real time" refers to a function that instantly converts what the veterinarian says into text format and sends that text to the system.

[1468] An "operation indicating the start of examination" is a specific action taken by the veterinarian to start the examination, and includes, for example, pressing a button or inputting a voice command.

[1469] The present invention provides a system for streamlining the examination process by veterinarians and improving the accuracy of diagnoses. The system includes: means for converting speech from a veterinarian during an examination into text in real time using speech recognition technology; means for automatically generating an electronic medical record based on the converted text; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; means including an emotion engine for analyzing the content and speech patterns of the veterinarian's speech and recognizing emotions; and means for adjusting the priority of diagnosis candidates based on the emotion information.

[1470] First, the user (veterinarian) starts the examination on the device. This can be done by pressing the start examination button or using a specific voice command. When the device detects this operation, it sends a notification that the examination has started to the server. The server then sends instructions to the device to start the voice recognition system and emotion engine.

[1471] While examining an animal, the user (veterinarian) verbally explains the animal's condition and observations. For example, the user might say, "This dog has had no appetite recently and seems to be listless." The device records this comment in real time and converts the voice data into text. The converted text data is immediately sent to the server.

[1472] The server generates an electronic medical record based on the received text data. Keywords extracted from the voice text and details of the examination are automatically recorded in the electronic medical record. For example, symptoms such as "loss of appetite" and "lack of energy" are recorded. Furthermore, the emotion engine analyzes the content of the veterinarian's speech and voice patterns to recognize the veterinarian's emotions. For example, it can detect when the veterinarian is anxious or suspicious. This emotional information is added to the text data and recorded in the electronic medical record.

[1473] Next, the server automatically analyzes possible disease names and treatment plans based on the contents of the electronic medical record and emotional information, and presents the candidates on the terminal. The presented candidates are displayed in an interface that allows the user (veterinarian) to select one while observing the animal's condition. The user (veterinarian) selects the most appropriate one from the presented candidates and explains it to the owner. If the emotion engine detects the veterinarian's anxiety, the priority of the disease name and treatment plan candidates is adjusted to match that anxiety. This allows veterinarians to access the information they need quickly, improving the accuracy of diagnoses.

[1474] Finally, based on the selected treatment plan, the device automatically generates an appropriate prescription, which is tailored to the animal's species and size and stored on the server, allowing veterinarians to create accurate prescriptions without any hassle.

[1475] As a concrete example, consider the case of a veterinarian examining a dog. The veterinarian begins the examination, and the device activates the voice recognition system and emotion engine. When the veterinarian says, "This dog has no appetite and is listless," this is converted into text in real time and sent to the server. If the emotion engine detects a sense of concern in the veterinarian's voice, this information is also added to the text data. Based on this information, the server records the information "lack of appetite" and "listlessness" in the patient's chart and presents possible illnesses such as "gastrointestinal upset" and "infection" along with a treatment plan. The veterinarian selects "gastrointestinal upset" and gives the owner an automatically generated prescription based on this information.

[1476] An example prompt might be: "Please explain in detail, step by step, how your system would respond when the veterinarian says, 'This dog has no appetite and is listless.'"

[1477] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1478] Step 1:

[1479] The user (veterinarian) starts the examination on the terminal.

[1480] Input: Pressing the Start Visit button or a specific voice command.

[1481] Specific operation: The user presses the "Start consultation" button on the device's touchscreen or says the voice command "Start consultation."

[1482] Output: Consultation start event.

[1483] Step 2:

[1484] The terminal detects the start of a medical examination event and transmits a notice of the start of a medical examination to the server.

[1485] Input: Consultation start event.

[1486] Specific operation: The device sends an HTTP request to the server triggered by the start of a consultation event, which includes metadata such as the timestamp of the consultation start and the veterinarian's ID.

[1487] Output: Notification to the server that consultation has started.

[1488] Step 3:

[1489] The server receives notification that the consultation has started and sends instructions to the terminal to activate the voice recognition system and emotion engine.

[1490] Input: Notification of start of consultation from terminal.

[1491] Specific operation: As an HTTP response, the server sends a message to the terminal containing commands to start the voice recognition system and emotion engine.

[1492] Output: Instructions to the device to start the voice recognition system and emotion engine.

[1493] Step 4:

[1494] The user (veterinarian) examines the animal and verbally explains the animal's condition and observation results.

[1495] Input: Veterinarian statement.

[1496] Specific actions: The user (veterinarian) verbally explains the results of the examination, such as, "This dog has had no appetite recently and seems to be in a bad mood."

[1497] Output: The veterinarian's remarks are recorded as audio data.

[1498] Step 5:

[1499] The device records what the veterinarian says as audio data in real time and converts the audio data into text.

[1500] Input: Veterinarian speech data.

[1501] How it works: A speech recognition system converts speech data into text using machine learning models.

[1502] Output: Text data.

[1503] Step 6:

[1504] The terminal transmits the generated text data to the server.

[1505] Input: Text data.

[1506] Specific operation: The generated text data is packaged in JSON format and sent to the server as an HTTP request.

[1507] Output: Sends text data to the server.

[1508] Step 7:

[1509] The server generates an electronic medical record based on the received text data.

[1510] Input: Text data.

[1511] Specific operation: The server analyzes the received text data, identifies important medical details using a keyword extraction algorithm, and records them in the electronic medical record.

[1512] Output: Generated electronic medical record.

[1513] Step 8:

[1514] The server works in conjunction with the text data to run an emotion engine and analyze the veterinarian's emotions.

[1515] Input: Audio data.

[1516] How it works: The emotion engine uses machine learning models to analyze the tone, rate, emphasis, etc. of the voice and infer the veterinarian's emotional state.

[1517] Output: Emotional information.

[1518] Step 9:

[1519] Emotional information is added to the text data and recorded in the electronic medical record.

[1520] Input: Emotion information.

[1521] Specific operation: The veterinarian's emotion label is added to the text data and sent back to the server.

[1522] Output: Electronic medical record with emotional information added.

[1523] Step 10:

[1524] The server automatically analyzes the disease name and possible treatment options based on the contents of the electronic medical record and emotional information, and presents them on the terminal.

[1525] Input: Electronic medical record and emotional information.

[1526] How it works: The server analyzes the EMR and emotion information, uses machine learning models to search for relevant cases in the database, and sends a prioritized list of potential diagnoses and treatment options to the device.

[1527] Output: A list of potential diagnoses and treatment strategies.

[1528] Step 11:

[1529] The terminal displays the presented candidates in an interface that allows the user (veterinarian) to select one.

[1530] Input: A list of potential diagnoses and treatment strategies.

[1531] Specific operation: The terminal visually displays diagnostic candidates and treatment plans, and provides an interface that allows the user to easily make selections.

[1532] Output: User selection of potential diagnoses and treatment strategies.

[1533] Step 12:

[1534] The user (veterinarian) selects a candidate diagnosis, and the terminal automatically generates an appropriate prescription based on that.

[1535] Input: Selected candidate diagnoses.

[1536] Specific operation: The terminal generates a prescription based on the selected diagnostic candidate according to the type and size of the animal and stores it on the server.

[1537] Output: Auto-generated prescription.

[1538] (Application example 2)

[1539] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1540] The modern veterinarian consultation process requires accurate recording of consultation data, highly accurate diagnosis, and efficient treatment decisions. However, current methods require all of these processes to be done manually, which is not only time-consuming and labor-intensive, but also makes the accuracy of diagnosis highly dependent on the veterinarian's experience. Furthermore, as the demand for remote consultations increases, there is a need to provide consultations of the same quality remotely. To solve these challenges, a system that integrates speech recognition technology, emotion analysis technology, natural language processing technology, and remote consultation functionality is required.

[1541] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for converting the doctor's utterances during the consultation into text in real time using voice recognition technology; means for automatically generating an electronic medical record based on the converted text; means for analyzing the doctor's emotional state using emotion analysis technology and adding it to the text data; means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record and emotion information; means for the doctor to select an appropriate disease name and treatment plan from the presented disease names and treatment plans; means for automatically generating a prescription based on the selected treatment plan; and means having a real-time video call function for remote consultations. This realizes efficient examination processes and improved diagnostic accuracy, making it possible to provide the same quality as conventional methods even in remote consultations.

[1542] "Speech recognition technology" is a technology that converts voice data into text data.

[1543] "Emotion analysis technology" is a technology that analyzes and determines a speaker's emotional state from voice and text data.

[1544] An "electronic medical record" is digital data used to manage the medical records of patients and animals on an information system, replacing paper medical records.

[1545] "Natural language processing technology" is a technology that allows computers to understand and process natural language.

[1546] "Remote consultation" is a method in which a doctor can conduct an examination via the Internet while being in a remote location from the patient or owner.

[1547] "Candidates for diagnosis and treatment" is a list of possible diagnoses and treatment methods based on the examination.

[1548] The "real-time video call function" is a function that allows two-way communication of video and audio in real time via the Internet.

[1549] A "prescription" is a document that lists specific medications or treatments prescribed by a physician for a patient or animal.

[1550] The present invention provides a system for improving the efficiency of the process when a veterinarian examines an animal and improving the accuracy of diagnosis. Specific embodiments are described below.

[1551] System configuration

[1552] 1. Use of voice recognition technology

[1553] The system uses speech recognition technology to convert what the veterinarian says into text in real time, using the Google Cloud Speech-to-Text API or a similar speech recognition API.

[1554] 2. Use of emotion analysis technology

[1555] Use IBM Watson Tone Analyzer or a similar sentiment analysis engine to analyze the veterinarian's speech and emotional state from their voice patterns.

[1556] 3. Generating Electronic Medical Records

[1557] Based on the converted text data and emotion information, an electronic medical record is automatically generated. The medical records are stored in a database system (e.g., MongoDB).

[1558] 4. Implementing real-time video calling

[1559] Incorporate video calling technology (e.g., WebRTC) to allow owners to receive consultations remotely.

[1560] 5. Diagnosis and treatment plan

[1561] Based on the generated electronic medical record and emotional information, symptoms are analyzed using natural language processing technology (e.g., generative AI model), and appropriate disease names and treatment options are generated and presented to the veterinarian.

[1562] 6. Automatic prescription generation

[1563] Based on the diagnostic results selected by the veterinarian, the appropriate prescription is automatically generated and provided to the owner in digital format.

[1564] Specific examples of functions

[1565] Process during consultation:

[1566] 1. Launching speech recognition and emotion analysis

[1567] When the veterinarian presses the "Start Examination" button, the system simultaneously activates voice recognition and emotion analysis technologies.

[1568] 2. Processing of veterinarians' speech data

[1569] When a veterinarian says, "This dog has been coughing a lot recently and seems lethargic," the voice recognition system converts this statement into text data in real time.

[1570] The emotion analysis engine detects emotions such as "worry" from the veterinarian's comments and adds them to the text data.

[1571] 3. Automatic generation of electronic medical records

[1572] Based on the converted text data and emotional information, content such as "Symptoms: cough, lethargy" and "Veterinarian's emotions: worried" are automatically recorded in the electronic medical record.

[1573] 4. Providing diagnosis and treatment guidelines

[1574] The system references electronic medical records and emotional information, and uses a "generative AI model" to suggest possible disease names such as "bronchitis" and "pneumonia," as well as treatment options.

[1575] The veterinarian will confirm the diagnosis based on the options presented.

[1576] 5. Automatic prescription generation

[1577] Based on the confirmed diagnosis, a specific prescription such as "antibiotics (300 mg) twice a day for 7 days" is automatically generated.

[1578] Example prompt sentence:

[1579] "Examination: This dog has been coughing a lot recently and seems lethargic.

[1580] Veterinarian's emotional state: Anxiety.

[1581] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[1582] In this way, the system supports veterinarians throughout the entire consultation process, enabling efficient and highly accurate consultations.

[1583] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1584] Step 1:

[1585] When the user (veterinarian) presses the button to start the consultation, the server simultaneously launches the speech recognition technology and emotion analysis technology. The input is the operation signal to start the consultation, and based on this, the server initializes the speech recognition API (e.g., Google Cloud Speech-to-Text) and emotion analysis engine (e.g., IBM Watson Tone Analyzer). The output is a state in which these technologies are ready to operate.

[1586] Step 2:

[1587] The device uses real-time video calling functionality to transmit video and audio between the owner and the veterinarian in two directions. The input is the audio and video data of the owner and the veterinarian, which is processed in real time through a video calling library (e.g., WebRTC). The output is a real-time video call session.

[1588] Step 3:

[1589] The device uses a voice recognition system to convert what the veterinarian says during the examination into text in real time. The input is the veterinarian's voice data, which is analyzed by the voice recognition API to generate text data. The output is the text data of the veterinarian's speech.

[1590] Step 4:

[1591] The server analyzes the text data using an emotion analysis engine to analyze the veterinarian's emotional state. The input is the text data of the veterinarian's comments, which the emotion analysis engine analyzes and adds emotional information. The output is text data with the added emotional information.

[1592] Step 5:

[1593] The server automatically generates an electronic medical record based on text data with added emotional information. The input is text data and emotional information, which are recorded in a database system (e.g., MongoDB) to generate the electronic medical record. The output is the electronic medical record data.

[1594] Step 6:

[1595] The server uses natural language processing technology to analyze symptoms using electronic medical record data and emotional information. The input is the electronic medical record data and emotional information, and by invoking the generative AI model with a prompt sentence, it generates appropriate disease names and treatment plan candidates. The output is disease names and treatment plan candidates.

[1596] Step 7:

[1597] The terminal presents the generated disease name and treatment plan candidate list to the veterinarian, allowing them to select the most appropriate diagnosis. The disease name and treatment plan candidate list are input, and the veterinarian makes the selection via the interface. The selected diagnosis result is output.

[1598] Step 8:

[1599] The server automatically generates an appropriate prescription based on the selected diagnostic results and provides it to the owner in digital form. The input is the selected diagnostic results, and the prescription data is generated based on this. The output is a digital prescription.

[1600] Specific actions

[1601] Example prompt sentence:

[1602] "Examination: This dog has been coughing a lot recently and seems lethargic.

[1603] Veterinarian's emotional state: Anxiety.

[1604] Based on the analysis results, please suggest an appropriate disease name and treatment plan.

[1605] Through this series of steps, the system streamlines the entire consultation process and enables highly accurate diagnosis and treatment.

[1606] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1607] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1608] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1609] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1610] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1611] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1612] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1613] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1614] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1615] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1616] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1617] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1618] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1619] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1620] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1621] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1622] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1623] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1624] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1625] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1626] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1627] The following is further disclosed regarding the above embodiment.

[1628] (Claim 1)

[1629] A method that uses voice recognition technology to convert what a doctor says during an examination into text in real time, and

[1630] a means for automatically generating an electronic medical record based on the converted text;

[1631] A means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record;

[1632] A means for a doctor to select an appropriate disease name and treatment plan from the presented ones;

[1633] means for automatically generating a prescription based on the selected course of treatment;

[1634] A system including:

[1635] (Claim 2)

[1636] 2. The system according to claim 1, further comprising means for automatically activating voice recognition technology when a doctor performs an operation indicating the start of a medical examination.

[1637] (Claim 3)

[1638] The system according to claim 1, further comprising means for analyzing symptoms from information recorded in an electronic medical record using natural language processing technology and generating appropriate disease names and treatment plan candidates.

[1639] "Example 1"

[1640] (Claim 1)

[1641] A method that uses voice recognition technology to convert what a doctor says during an examination into text in real time, and

[1642] means for automatically generating an electronic medical record based on the converted text;

[1643] A means for automatically presenting disease names and treatment plan candidates from the generated electronic medical records;

[1644] A means for a doctor to select an appropriate disease name and treatment plan from the presented ones;

[1645] means for automatically generating a prescription based on the selected course of treatment;

[1646] A system including:

[1647] (Claim 2)

[1648] 2. The system according to claim 1, further comprising means for automatically activating voice recognition technology when a doctor performs an operation indicating the start of a medical examination.

[1649] (Claim 3)

[1650] The system according to claim 1, further comprising means for analyzing symptoms from information recorded in an electronic medical record using natural language processing technology and generating appropriate disease names and treatment plan candidates.

[1651] "Application Example 1"

[1652] (Claim 1)

[1653] A method that uses voice recognition technology to convert what a doctor says during an examination into text in real time, and

[1654] a means for automatically generating an electronic medical record based on the converted text;

[1655] A means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record;

[1656] A means for a doctor to select an appropriate disease name and treatment plan from the presented ones;

[1657] means for automatically generating a prescription based on the selected course of treatment;

[1658] A means of converting conversations with customers in physical stores into text in real time using voice recognition technology and presenting product information and recommended products based on the conversation;

[1659] A system including:

[1660] (Claim 2)

[1661] 2. The system according to claim 1, further comprising means for automatically activating voice recognition technology when a doctor performs an operation indicating the start of a medical examination.

[1662] (Claim 3)

[1663] The system according to claim 1, further comprising means for analyzing symptoms from information recorded in an electronic medical record using natural language processing technology and generating appropriate disease names and treatment plan candidates.

[1664] (Claim 4)

[1665] The system described in claim 1 further includes a means for generating product descriptions and recommended product information in the form of prompt sentences using a generative AI model based on the results of speech recognition and text conversion in a physical store, and presenting the information to store clerks via voice.

[1666] "Example 2: Combining Emotion Engines"

[1667] (Claim 1)

[1668] A method that uses voice recognition technology to convert the veterinarian's speech during an examination into text in real time,

[1669] a means for automatically generating an electronic medical record based on the converted text;

[1670] A means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record;

[1671] A means for veterinarians to select the appropriate disease name and treatment plan from the presented ones, and

[1672] means for automatically generating a prescription based on the selected course of treatment;

[1673] means including an emotion engine that analyzes speech content and speech patterns of the veterinarian to recognize emotions;

[1674] a means for adjusting the priorities of candidate diagnoses based on emotional information;

[1675] A system including:

[1676] (Claim 2)

[1677] The system according to claim 1, further comprising means for automatically activating the voice recognition technology and emotion engine when a veterinarian performs an operation indicating the start of a medical examination.

[1678] (Claim 3)

[1679] The system according to claim 1, further comprising means for analyzing symptoms using natural language processing technology from information recorded in an electronic medical record and detected emotional information, and generating appropriate disease names and treatment plan candidates.

[1680] "Application example 2 when combining emotion engines"

[1681] (Claim 1)

[1682] A method that uses voice recognition technology to convert what a doctor says during an examination into text in real time, and

[1683] a means for automatically generating an electronic medical record based on the converted text;

[1684] A means for analyzing the emotional state of the doctor using emotion analysis technology and adding the analyzed emotional state to the text data;

[1685] A means for automatically presenting disease names and treatment plan candidates from the generated electronic medical records and emotional information;

[1686] A means for a doctor to select an appropriate disease name and treatment plan from the presented ones;

[1687] means for automatically generating a prescription based on the selected course of treatment;

[1688] A means having a real-time video call function for remote consultation;

[1689] A system including:

[1690] (Claim 2)

[1691] The system according to claim 1, further comprising means for automatically activating the voice recognition technology and the emotion analysis technology when the doctor performs an operation indicating the start of the consultation.

[1692] (Claim 3)

[1693] The system according to claim 1, characterized in that it includes a means for analyzing symptoms using natural language processing technology from information recorded in the electronic medical record and sentiment analysis information, and generating appropriate disease names and treatment plan candidates. [Explanation of symbols]

[1694] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A method that uses voice recognition technology to convert what a doctor says during an examination into text in real time, and a means for automatically generating an electronic medical record based on the converted text; A means for automatically presenting disease names and treatment plan candidates from the generated electronic medical record; A means for a doctor to select an appropriate disease name and treatment plan from the presented ones; means for automatically generating a prescription based on the selected course of treatment; A system including:

2. 2. The system according to claim 1, further comprising means for automatically activating the voice recognition technology when the doctor performs an operation indicating the start of a medical examination.

3. 2. The system according to claim 1, further comprising means for analyzing symptoms from information recorded in an electronic medical record using natural language processing technology and generating appropriate disease names and treatment plan candidates.

Images