Program, information processing device and method

Abstract

To provide a program for reducing labor for recording electronic medical records input on the basis of voice data, and at the same time, delivering mass-customized data to medical institutions throughout Japan.SOLUTION: A program for operating a computer including a processor and a memory, where the memory stores a machine learning model that receives speech recognition data including at least one of Hiragana, Katakana, alphanumeric characters and punctuation marks as input and outputs Kanji-converted speech recognition data after performing kana-Kanji conversion on the speech recognition data, the program: causes the processor to receive input of speech data from a user, perform speech recognition on the received speech data to acquire speech recognition data, and input the acquired speech recognition data to the machine learning model; and causes the machine learning model to output Kanji-converted speech recognition data corresponding to the input speech recognition data, and present the Kanji-converted speech recognition data to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing device, and a method. [Background technology]

[0002] Electronic medical records are well known in which a doctor electronically records the contents of a patient's medical interview and the results of the interview by voice or keyboard, and also electronically records the history of medical procedures performed on the patient. The contents of the electronic medical record may be created according to an electronic medical record template.

[0003] A technique related to the above-mentioned technique is disclosed in Patent Document 1.

[0004] Patent Document 1 discloses technology related to a medical support device. In the medical support device, an input item display means displays input items on a display. An input item selection means selects one of the input items. A speech recognition means uses a selected dictionary to perform speech recognition on the input speech and extracts word candidates for the speech. A word candidate display means displays the extracted word candidates on a display. A selection operation acceptance means accepts an operation to select one word candidate from the word candidates. A memory control means stores the selected word candidate in a memory means as an answer to the selected one of the input items. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-156844 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology described in Patent Document 1 performs speech recognition processing using a specialized dictionary for the medical field, but since template input in the medical field often involves homonyms with different spellings, there are certain limitations to the accuracy of speech recognition processing even at present, and doctors and others may need to make corrections to the content entered into the electronic medical record template as a result of the speech recognition processing.

[0007] Therefore, the present disclosure has been made to solve the above problem, and its purpose is to provide a technology that reduces the labor required for recording input into an electronic medical record template based on voice data. [Means for solving the problem]

[0008] A program for operating a computer having a processor and a memory. The memory stores a machine learning model that receives speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input and outputs kanji-converted speech recognition data by performing kana-to-kanji conversion on the speech recognition data. The program causes the processor to execute a thirteenth step of accepting input of utterance data from a user, a fourteenth step of acquiring speech recognition data by performing speech recognition on the accepted utterance data in the thirteenth step, a fifteenth step of inputting the speech recognition data acquired in the fourteenth step into the machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and a sixteenth step of presenting the kanji-converted speech recognition data, which is the output of the fifteenth step, to the user. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to reduce the labor required for recording input data into an electronic medical record template based on voice data. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an overall configuration of a system according to an embodiment. [Figure 2]FIG. 2 is a diagram illustrating a functional configuration of a terminal device according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating a functional configuration of a server according to an embodiment. [Figure 4] FIG. 2 is a diagram showing the data structure of an electronic medical record database according to one embodiment. [Figure 5] 10 is a flowchart illustrating an example of a processing flow in a system according to an embodiment. [Figure 6] 10 is a flowchart illustrating another example of the processing flow in the system according to an embodiment. [Figure 7] FIG. 10 is a schematic diagram illustrating an example of a screen displayed on a terminal device according to an embodiment. [Figure 8] FIG. 1 is a diagram illustrating an example of a processing flow in a system according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating another example of the processing flow in the system according to an embodiment. [Figure 10] FIG. 10 is a diagram showing yet another example of the processing flow in the system according to an embodiment. [Figure 11] FIG. 10 is a schematic diagram illustrating another example of a screen displayed on the terminal device according to an embodiment. [Figure 12] FIG. 10 is a schematic diagram illustrating yet another example of a screen displayed on a terminal device according to an embodiment. [Figure 13] FIG. 10 is a schematic diagram illustrating yet another example of a screen displayed on a terminal device according to an embodiment. [Figure 14] FIG. 10 is a schematic diagram illustrating yet another example of a screen displayed on a terminal device according to an embodiment. [Figure 15] FIG. 10 is a schematic diagram illustrating yet another example of a screen displayed on a terminal device according to an embodiment. [Figure 16] FIG. 10 is a schematic diagram illustrating yet another example of a screen displayed on a terminal device according to an embodiment. [Figure 17] FIG. 10 is a diagram illustrating a procedure for generating a letter of introduction by a system according to an embodiment. [Figure 18] FIG. 10 is a diagram illustrating a procedure for generating a letter of introduction by a system according to an embodiment. [Figure 19] 10 is a flowchart showing an example of a processing flow in a system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated description will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0012] In the following description, a "processor" refers to one or more processors. The at least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may also be another type of processor such as a GPU (Graphics Processing Unit). The at least one processor may be single-core or multi-core.

[0013] Furthermore, the at least one processor may be a processor in the broad sense, such as a hardware circuit (for example, a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) that performs part or all of the processing.

[0014] In the following explanation, information that produces an output for an input may be described using expressions such as "xxx table," but this information may be data of any structure, or may be a learning model such as a neural network that produces an output for an input. Therefore, an "xxx table" may be referred to as "xxx information."

[0015] Furthermore, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0016] In addition, in the following explanation, processing may be described using the "program" as the subject, but since a program is executed by a processor to perform specified processing while appropriately using a memory unit and / or an interface unit, etc., the subject of the processing may also be the processor (or a device such as a controller that has that processor).

[0017] The program may be installed in a device such as a computer, or may be stored in, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. Also, in the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0018] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs, CPUs, conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes programs stored in memory.

[0019] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0020] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0021] Furthermore, in the following description, identification numbers are used as identification information for various objects, but other types of identification information (for example, identifiers including alphabetic characters or symbols) may also be used.

[0022] In addition, in the following description, when describing elements of the same type without distinguishing between them, reference symbols (or common symbols among the reference symbols) may be used, and when describing elements of the same type with distinction between them, the identification numbers (or reference symbols) of the elements may be used.

[0023] In the following description, the control lines and information lines are those that are considered necessary for the description, and do not necessarily represent all the control lines and information lines in the product. All components may be interconnected.

[0024] <0 System Overview> The system according to the present disclosure is a system that uses speech recognition to record the contents of an electronic medical record based on an electronic medical record template. In this specification, the contents of an electronic medical record are generated based on the electronic medical record template.

[0025] An electronic medical record template is structured data that has input fields and input content associated with these input fields. Here, structured data is data that is predefined and formatted to have a certain structure before being placed in storage. In contrast, unstructured data is data that is stored in plain text and is not processed until it is used. The input fields of an electronic medical record are defined based on this electronic medical record template. Note that this system also includes a system that assists in the input of template input fields using an electronic medical record template input assistance system that recreates the input fields of an electronic medical record template in a different format using a web form, etc.

[0026] Input items correspond to each item in the electronic medical record, and are relatively short pieces of information using medically designated terms so that medical professionals can identify which item it is. Input content is what a doctor or medical professional enters into the input item associated with this input content. Input content can be in multiple choice format or free-response format, and the question format can vary. If the input content is in multiple choice format, the question is made by selecting one of the options (sometimes there is only one option), and if it is free-response format, free text is entered. Note that if the input content is in multiple choice format, the options include medically designated terms. Electronic medical record template data is what a doctor or medical professional enters based on the electronic medical record template, and is the specific content of the input content of the electronic medical record template.

[0027] In medical settings, doctors and medical staff need to input a large amount of data into electronic medical records based on electronic medical record templates, which are structured data, some of which are multiple choice, and some of which are free-response.

[0028] The input fields and input content of electronic medical record templates are created with the assumption that they will be entered, modified, and added to by medical professionals, and that they will also be viewed by medical professionals. This means that the input fields and input content require medical knowledge and must be medically accurate. However, the amount of information that medical professionals, including doctors, must record in electronic medical records is enormous, and the effort required is enormous. For example, at one medical facility's admission and discharge support center, there were approximately six pages of input content. Finding which field on the electronic medical record's long profile section or assessment sheet to enter the information into and entering it there took approximately 20 minutes per patient, resulting in overtime work for nurses and medical office staff.

[0029] From this perspective, it is conceivable to record the contents of electronic medical records using voice recognition. However, even at present, the accuracy of voice recognition processing is not sufficient. In particular, the contents to be recorded in electronic medical records include patient names, designated medical terms, and other content that is unlikely to be accurately converted from kana to kanji using a general voice recognition engine.

[0030] Therefore, in the system according to the present disclosure, when recording the contents of an electronic medical record based on an electronic medical record template, a machine learning model performs speech recognition processing. Preferably, medical information such as case information and medical notes is input to the machine learning model as a prompt, and the contents of the electronic medical record are identified using the output from the machine learning model. By adopting such a configuration, the contents of the electronic medical record can be recorded efficiently and accurately using speech recognition technology.

[0031] Here, case information refers to information about a specific patient (not limited to a single person) that includes at least the current symptoms, and preferably includes the patient's age, sex, chief complaint heard by the medical professional, current medical history (history of present illness), past medical history, family history, findings at the first visit and test values ​​for the patient, patient outcomes, imaging test findings, and audio information.Case information includes information interviewed by a medical professional in person, information entered by the patient on a medical questionnaire, and audio recordings.

[0032] In order to input case information into the system, it must be converted into electronic data (e.g., text data). For example, if a medical professional interviews a patient in person, the details of the interview that the medical professional manually enters into the free-form fields of the electronic medical record will be included in the electronic case information. In addition, the details entered by the patient into the medical questionnaire prior to the medical professional's interview must also be digitized. In recent years, systems have become available that use tablet devices and other devices to electronically input answers to medical questionnaire questions, known as electronic medical questionnaires, and the answers to medical questionnaire questions on electronic medical questionnaires have already been digitized.

[0033] The system according to the present disclosure uses a machine learning model that receives speech recognition data containing at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input, and outputs kanji-converted speech recognition data after performing kana-kanji conversion on the speech recognition data. This machine learning model may be a general-purpose large-scale language model such as ChatGPT, or may be a machine learning model tuned for the system according to the present disclosure.

[0034] When inputting medical information into a machine learning model, it is preferable to input the medical information as a prompt. Of course, text other than medical information may be added to the medical information and input to the machine learning model. Details of the prompt will be described later.

[0035] However, just like with human operators, the accuracy of voice recognition is not 100%. The fact that it is easier than other methods to check whether the input was correct also has a significant impact on ease of use for users.

[0036] Furthermore, the system according to the present disclosure generates document data based on the electronic medical record data (including the structured data of the electronic medical record template) whose recorded content has been finalized by the above-mentioned operations, preferably the electronic medical record data for a specific patient, in accordance with instructions from a medical professional. The document data referred to here is, for example, data of documents generated using part of the electronic medical record data, such as a medical summary, a referral letter, or a report to a pharmaceutical company.

[0037] In the system according to the present disclosure, document templates for these document data are generated using a machine learning model. The machine learning model takes medically designated terms included in the input fields of the electronic medical record template as input and outputs a document template that includes these medically designated terms as character strings that are unlikely to conflict. This machine learning model may be a general-purpose large-scale language model such as ChatGPT, or it may be a machine learning model tuned for the system according to the present disclosure.

[0038] <One embodiment> <1 Overall system configuration> FIG. 1 is a diagram showing the overall configuration of an electronic medical record system 1 according to this embodiment. As shown in FIG. 1, the electronic medical record system 1 includes a plurality of terminal devices (terminal device 10A and terminal device 10B are shown in FIG. 1; hereinafter, they may be collectively referred to as "terminal device 10") and a server 20. The terminal devices 10 and the server 20 are connected to each other via a network 80 so that they can communicate with each other. The network 80 is configured as a wired or wireless network. In this embodiment, the server 20 is a server that functions as a web server (including a cloud server), and exchanges information with the terminal device 10 via web pages. In addition, a web page browser for viewing web pages is installed on the terminal device 10, but a dedicated application for providing the services of the server 20 may also be installed so that the web pages can be viewed using the dedicated application.

[0039] The terminal device 10 is realized by a desktop PC (Personal Computer), a laptop PC, etc. Alternatively, the terminal device 10 may be a tablet compatible with a mobile communication system, a mobile terminal such as a smartphone, etc.

[0040] The terminal device 10 is a device operated by a medical professional or an administrator of the electronic medical record system 1. Here, the term "medical professional" is a concept that includes doctors, nurses, medical technicians, etc. In the following explanation, the term "medical professional" will be taken to include the administrator of the system 1, except when a distinction is made between medical professionals and the administrator of the system 1.

[0041] A medical professional uses the terminal device 10 to record the contents of an electronic medical record based on an electronic medical record template. At this time, the medical professional inputs speech data into the terminal device 10 and instructs the input / correction / addition. The terminal device 10 sends the speech data to the server 20, requests speech recognition processing and kana-to-kanji conversion processing (natural language processing) by the server 20, and obtains kanji-converted speech recognition data from the server 20. Next, the medical professional inputs / corrects / adds the record contents based on the kanji-converted speech recognition data using the terminal device 10. Then, the medical professional instructs the terminal device 10 to record the input / correction / additional input contents as the electronic medical record contents.

[0042] The terminal device 10 is communicatively connected to the server 20 via a network 80. The terminal device 10 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 conforming to communication standards such as 4G, 5G, and LTE (Long Term Evolution), and a wireless LAN router 82 conforming to a wireless LAN (Local Area Network) standard such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. As shown in FIG. 1 , the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

[0043] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device (e.g., a keyboard, a touch panel, a touch pad, a pointing device such as a mouse, etc.) for receiving input operations from a user. The output device 14 is an output device (e.g., a display, a speaker, etc.) for presenting information to a user. The memory 15 is for temporarily storing programs and data processed by the programs, etc., and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for saving data, such as a flash memory or an HDD (Hard Disc Drive). The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0044] The server 20 is managed by an administrator of the electronic medical record system 1 of this embodiment, and stored contents are modified / added / deleted as appropriate by medical professionals who are users of the terminal device 10. The server 20 is an electronic medical record device, and medical professionals at medical facilities view the input items and input contents of the electronic medical record via the terminal device 10 and modify / add to the input contents. The server 20 also accepts editing operations of the electronic medical record template performed by the medical professionals via the terminal device 10, and modifies / adds / deletes the electronic medical record template based on these editing operations.

[0045] The server 20 is a computer connected to a network 80. The server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29.

[0046] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices. The input / output IF 23 functions as an interface with an input device for receiving input operations from a user and an output device for presenting information to the user. The memory 25 is for temporarily storing programs and data processed by the programs, etc., and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 26 is a storage device for saving data, such as a flash memory or an HDD (Hard Disc Drive). The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0047] <1.1 Functional configuration of the terminal device 10> FIG. 2 is a block diagram showing the functional configuration of a terminal device 10 constituting the system 1 of this embodiment. As shown in FIG. 2, the terminal device 10 includes a plurality of antennas (antenna 111, antenna 112), wireless communication units (first wireless communication unit 121, second wireless communication unit 122) corresponding to the respective antennas, an input device 13 (including a keyboard 131 and a mouse 132), an audio processing unit 17 (including a microphone 171 and a speaker 172), a display 141 as an output device 14, a storage unit 180, and a control unit 190. The terminal device 10 also has functions and configurations not specifically shown in FIG. 2 (e.g., a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in FIG. 2, the blocks included in the terminal device 10 are electrically connected by a bus or the like.

[0048] The antenna 111 emits a signal emitted by the terminal device 10 as a radio wave. The antenna 111 also receives a radio wave from space and provides the received signal to the first radio communication unit 121.

[0049] The antenna 112 emits a signal emitted by the terminal device 10 as a radio wave. The antenna 112 also receives a radio wave from space and provides the received signal to the second radio communication unit 122.

[0050] The first wireless communication unit 121 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal device 10 can communicate with other wireless devices. The second wireless communication unit 122 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 112 so that the terminal device 10 can communicate with other wireless devices. The first wireless communication unit 121 and the second wireless communication unit 122 are communication modules including a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation / demodulation and frequency conversion of wireless signals transmitted and received by the terminal device 10, and provide the received signals to the control unit 190.

[0051] The input device 13 has a mechanism for accepting input operations from the user. Specifically, the input device 13 includes a keyboard 131 and a mouse 132. Note that the input device 13 may be configured as a touch screen that detects the position of a user's touch on a touch panel, for example, by using a capacitive touch panel.

[0052] The keyboard 131 accepts input operations by the user of the terminal device 10. The keyboard 131 is a device for inputting characters, and outputs input character information to the control unit 190 as an input signal.

[0053] The mouse 132 accepts input operations by the user of the terminal device 10. The mouse 132 is a pointing device for selecting an object displayed on the display 141, and outputs position information of the selected object on the screen and information indicating that a button has been pressed as input signals to the control unit 190.

[0054] The audio processing unit 17 modulates and demodulates audio signals. The audio processing unit 17 modulates a signal provided from the microphone 171 and provides the modulated signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is realized, for example, by a processor for audio processing. The microphone 171 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 17. The speaker 172 converts the audio signal provided from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0055] Display 141 displays data such as images, videos, and text under the control of control unit 190. Display 141 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0056] Storage unit 180 is configured with, for example, a flash memory, and stores data and programs used by terminal device 10. In one aspect, storage unit 180 stores user information 182.

[0057] The user information 182 is information about a user who uses the terminal device 10 to confirm the input contents of the electronic medical record using voice recognition processing, which is a function of the system 1, and is preferably information about a medical professional who uses the system 1 of this embodiment.

[0058] The control unit 190 controls the operation of the terminal device 10 by reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 is, for example, an application that is pre-installed in the terminal device 10. The control unit 190 operates in accordance with an application program 181 stored in the storage unit 180, thereby fulfilling the functions of an input operation receiving unit 191, a transmission / reception unit 192, a data processing unit 193, and a presentation control unit 194.

[0059] The input operation receiving unit 191 performs processing to receive input operations by the user via an input device such as the keyboard 131 .

[0060] The transmitting / receiving unit 192 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol.

[0061] The data processing unit 193 performs calculations on data that the terminal device 10 has received as input in accordance with a program, and outputs the calculation results to a memory or the like.

[0062] The presentation control unit 194 performs processing to present information to the user. The presentation control unit 194 performs processing to display a display image on the display 141, processing to output sound to the speaker 172, and the like.

[0063] <1.2 Functional configuration of server 20> 3 is a diagram showing an example of the functional configuration of the server 20. As shown in FIG. 3, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0064] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0065] The memory unit 202 includes, for example, an electronic medical record DB 2022, electronic medical record data 2023, an electronic medical record template 2024, a document template 2025, speech data 2026, voice recognition data 2027, kanji-converted voice recognition data 2028, teacher data 2029, a machine learning model 2040, medical designated word data 2041, etc.

[0066] The electronic medical record DB 2022 is a database for managing electronic medical record data of patients who have visited a medical facility that uses the server 20. The electronic medical record DB 2022 may manage electronic medical record data from multiple medical facilities. Details will be described later.

[0067] The electronic medical record data 2023 is imported into the electronic medical record DB 2022 and becomes part of the recorded contents of the electronic medical record. The electronic medical record data 2023 has input items and input contents associated with these input items. There are no particular limitations on the data format of the electronic medical record data 2023, but the electronic medical record data 2023 in this embodiment is data written in XAML (Extensible Application Markup Language) converted into JSON (JavaScript Object Notation) format (JavaScript is a registered trademark). It is preferable that the input items of the electronic medical record data 2023 are assigned identifiers such as numeric strings, and these identifiers also constitute the electronic medical record data 2023. The electronic medical record data 2023 constitutes at least part of medical information including patient case information.

[0068] The electronic medical record template 2024 is a template used when generating the electronic medical record data 2023. The electronic medical record template 2024 is structured data that specifies input fields and the input content associated with these input fields. There are no particular limitations on the data format of the electronic medical record template 2024, but the electronic medical record template 2024 of this embodiment is data written in XAML that has been converted into JSON format, similar to the electronic medical record data 2023. As with the electronic medical record data 2023, it is preferable that the electronic medical record template 2024 has an identifier such as a numeric string assigned to its input fields, and this identifier also constitutes the electronic medical record template 2024.

[0069] Each electronic medical record template 2024 is associated with an identifier for identifying the respective electronic medical record template 2024. As an example, the identifier of the electronic medical record template 2024 is a numeric string of a predetermined number of digits.

[0070] In this embodiment, multiple answer candidates (candidates for input content) for medical interview questions may be associated with the input items in the electronic medical record template 2024. In other words, the input content may be one answer candidate selected from multiple answer candidates.

[0071] The document template 2025 is a template for document data generated by the server 20 of this embodiment. In this embodiment, document data is, for example, a medical summary, a referral letter, or a report to a pharmaceutical company, and a document template 2025 is provided for each type of document data.

[0072] The speech data 2026 is data obtained by recording speech input by a medical professional to the terminal device 10 via the microphone 171 of the voice processing unit 17, and is sent to the server 20 via the transmitting / receiving unit 192 of the control unit 190 of the terminal device 10. Preferably, the medical professional includes medical designated words included in medical designated word data 2041, which will be described later, in the speech data 2026. In other words, the medical professional inputs speech including medical designated words into the microphone 171 of the terminal device 10.

[0073] The voice recognition data 2027 is voice recognition data obtained as a result of a voice recognition module 2033 of the control unit 203, which will be described later, performing voice recognition based on the speech data 2026. The voice recognition data 2027 of this embodiment is data that includes a mixture of hiragana, katakana, alphanumeric characters, and punctuation marks, and preferably, blank data or the like is inserted as a delimiter between these hiragana, katakana, alphanumeric characters, and punctuation marks, making characters such as hiragana easier to read.

[0074] The kanji-converted speech recognition data 2028 is kanji-converted speech recognition data that is output by the machine learning model 2040 as a result of the later-described generative model input / output module 2035 inputting the speech recognition data 2027 into the machine learning model 2040. A method for obtaining the kanji-converted speech recognition data 2028 will be described in detail later.

[0075] The teacher data 2029 is teacher data that serves as learning data when the machine learning model 2040 is made to learn. The server 20 of this embodiment has two types of machine learning models 2040, and accordingly has teacher data 2029 for making each machine learning model 2040 learn.

[0076] One of the training data 2029 includes speech recognition data 2027 including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and kanji-converted speech recognition data 2028 obtained by performing kana-kanji conversion on the speech recognition data 2027. Preferably, the training data 2029 includes at least a portion of the electronic medical record data 2023.

[0077] The other training data 2029 has medical designated word data 2041 and a document template 2025 that includes the medical designated words included in the medical designated word data 2041 as character strings that are unlikely to collide.

[0078] One form of the machine learning model 2040 is a large-scale language model, such as ChatGPT, that receives input of a natural language or vector and outputs a natural language or vector corresponding to the natural language. In particular, the machine learning model 2040 of this embodiment undergoes machine learning using respective training data 2029. One trained machine learning model 2040 is then tuned to input speech recognition data 2027 containing at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and output kanji-converted speech recognition data 2028, which is subjected to kana-kanji conversion of the input speech recognition data 2027. Furthermore, the other trained machine learning model 2040 is also tuned to input medical designated terms contained in medical designated term data 2041 and output a document template 2025 containing the medical designated terms as character strings that are unlikely to collide.

[0079] Large-scale language models are generally referred to as generative AI, but they are created as models that use machine learning to replicate the generative process of human sentence creation. Large-scale language models are first trained with a large amount of pre-training data, then aligned based on instructions and on-site feedback, and the learning is completed to create a model. These are intended to improve the quality of the sentences generated by the generative AI, and also to create a model that can more accurately apply instructions during use, known as prompts, to the generation process. Once a model is completed, it is fixed. When using the model, instructions are given to the model via prompts to control the generation process. Note that obtaining and using text to enter in prompts using an external search engine is called RAG (Search Augmentation Generation).

[0080] The machine learning model 2040 according to this embodiment is, for example, a parameterized composite function formed by combining multiple functions. The parameterized composite function is defined by a combination of multiple adjustable functions and parameters. The prediction model according to this embodiment may be any parameterized composite function that satisfies the above requirements, but is assumed to be a multi-layer network model (hereinafter referred to as a multi-layer network). A prediction model using a multi-layer network has an input layer, an output layer, and at least one intermediate layer or hidden layer provided between the input layer and the output layer. The prediction model is expected to be used as a program module that is part of artificial intelligence software.

[0081] The multi-layered network according to the present embodiment may be, for example, a deep neural network (DNN), which is a multi-layered neural network that is the subject of deep learning. As the DNN, for example, a convolution neural network (CNN) that targets images may be used.

[0082] Furthermore, the above is merely an example of a prediction model, and the prediction model may have other configurations. For example, the prediction model may be a rule-based model described by a function in which chief complaint information and environmental information are used as variables and each variable is assigned a coefficient derived from past performance.

[0083] The medical designated word data 2041 is medical designated word data used by the electronic medical record data generation module 2037 of the control unit 203 to identify input items related to input content to be corrected / added when correcting / adding to the input content of the electronic medical record based on the kanji-converted speech recognition data 2028. The medical designated words referred to here are not limited to electronic medical records, but are terms generally used by medical professionals when writing information into medical records, and include at least so-called medical terms, as well as specialized terms used in input items. Preferably, the medical designated word data 2041 has a synonym dictionary for these medical designated words, etc., and is used to identify and maintain the linkage of items when correcting or updating. Medical terms are terms used in medical institutions to accurately describe medical terms, such as "medical history," "history of current illness," "medication history," "social history," "disease name," and "drug name." Synonyms for "history of current illness," such as "present symptom" and "HPI," may be used to guide or confirm the linkage when linking templates. The control unit 203 is realized by the processor 29 reading an application program 2021 stored in the storage unit 202 and executing instructions included in the application program 2021. By operating in accordance with the application program 2021, the control unit 203 performs functions shown as a reception control module 2031, a transmission control module 2032, a voice recognition module 2033, a prompt generation module 2034, a generation model input / output module 2035, a presentation control unit 2036, and an electronic medical record data generation module 2037.

[0084] The reception control module 2031 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0085] The transmission control module 2032 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0086] The voice recognition module 2033 acquires voice recognition data 2027 as an output result based on speech data 2026 input by the medical professional via the microphone 171. As described in the explanation of the voice recognition data 2027, the voice recognition module 2033 of this embodiment outputs voice recognition data 2027 that is a mixture of hiragana, katakana, alphanumeric characters, and punctuation marks, and preferably outputs voice recognition data 2027 in which blank data is inserted as delimiters between the hiragana, katakana, alphanumeric characters, and punctuation marks.

[0087] When various inputs are made to the machine learning model 2040 in the storage unit 202, the prompt generation module 2034 generates a prompt, which is text data to be input to the machine learning model 2040, and temporarily stores the generated prompt in the storage unit 202.

[0088] While there are no particular limitations on the prompts generated by the prompt generation module 2034, as an example, the prompt generation module 2034 of this embodiment may generate medical information related to the content of voice input by a medical professional. For example, if the voice input is for inputting electronic medical record template data for a specific patient, the generated prompt may be at least a portion of the electronic medical record template data for this specific patient that has been previously accumulated. Furthermore, the prompt generated by the prompt generation module 2034 may be text about the medical department to which the medical professional using the terminal device 10 belongs. The electronic medical record template data included in the prompt may include information (ID) for identifying the patient, previous electronic medical record data 2023, a medical summary from when the patient was hospitalized, and the like. These may be collectively referred to as the medical history of a specific patient. Furthermore, the prompts generated by the prompt generation module 2034 may also include prompts that optimize the output from the machine learning model 2040. Specific examples of such prompts will be described in detail below using screen examples, etc.

[0089] The generation model input / output module 2035 inputs the speech recognition data 2027 obtained as a result of the speech recognition processing performed by the speech recognition module 2033, and the prompts generated by the prompt generation module 2034, etc., into the machine learning model 2040, and causes this machine learning model 2040 to output kanji-converted speech recognition data 2028 corresponding to the speech recognition data 2027 input to the machine learning model 2040.

[0090] Alternatively, the generation model input / output module 2035 inputs the medical designated words contained in the medical designated word data 2041 and the prompts generated by the prompt generation module 2034 into the machine learning model 2040, and causes this machine learning model 2040 to output a document template 2025 that includes the medical designated words input into the machine learning model 2040 as strings of characters that are unlikely to collide.

[0091] At this time, the generative model input / output module 2035 generates document data by inserting structured data, using as a key a medically designated term included in the structured data of the electronic medical record template 2024 for the specific patient, into the document template 2025 output from the machine learning model 2040. The generated document data is temporarily stored in the storage unit 202.

[0092] The presentation control unit 2036, in cooperation with the transmission control module 2032, transmits the output results of the voice recognition module 2033, the prompt generation module 2034, the generation model input / output module 2035, and the electronic medical record data generation module 2037 described later to the terminal device 10 operated by a medical professional, and displays them on the display 141 of the terminal device 10.

[0093] At this time, the presentation control unit 2036 presents the recorded content to be recorded in the electronic medical chart template 2024, identified by the electronic medical chart data generation module 2037, and the electronic medical chart template 2024 other than this recorded content, by changing the presentation mode.

[0094] Alternatively, the presentation control unit 2036 presents the document template 2025, which is the output result of the machine learning model 2040, by changing the presentation manner between the structured data inserted into the document template 2025 by the generative model input / output module 2035 and document data other than the structured data inserted into the document template 2025.

[0095] The presentation control unit 2036 also presents a pair of input items of the structured data inserted into the document template 2025 by the generative model input / output module 2035 and the input content corresponding to these input items. At this time, the presentation control unit 2036 accepts a selection input of an input item and the input content corresponding to this input item to be inserted into the document data generated by the generative model input / output module 2035, from among the paired presented input items of the structured data inserted into the document template 2025 and the input content corresponding to these input items. The generative model input / output module 2035 inserts the input item for which the selection input was accepted and the input content corresponding to this input item into the document data.

[0096] The electronic medical record data generation module 2037 identifies the record content to be recorded in the electronic medical record template 2024 based on the medically designated words included in the kanji-converted speech recognition data 2028. Then, the electronic medical record data generation module 2037 adds or updates the electronic medical record template 2024 based on the identified record content.

[0097] <2 Data Structure> Fig. 4 is a diagram showing the data structure of a database stored in the server 20. Note that Fig. 4 is an example and does not exclude data not shown.

[0098] The database shown in Figure 4 is a relational database, which manages data sets called tables, which are structured by rows and columns, by associating them with each other. In a database, a table is called a table, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables can be set and associated.

[0099] Typically, each table has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit 203 of the server 20 can cause the processor 29 to add, delete, or update records in a specific table stored in the storage unit 202 according to various programs.

[0100] 4 is a diagram showing the data structure of the electronic medical record DB 2022. As shown in FIG. 4, each record of the electronic medical record DB 2022 includes, for example, an item "electronic medical record ID," an item "patient ID," an item "department ID," and an item "electronic medical record data." Each item of the electronic medical record DB 2022 is input by an electronic medical record template data recording module (not shown) when this electronic medical record template data recording module generates electronic medical record data 2023. The information stored in the electronic medical record DB 2022 can be changed and updated as appropriate.

[0101] The item "Electronic Medical Record ID" is an ID for identifying an electronic medical record managed by the system 1 (particularly the server 20) of this embodiment. The item "Patient ID" is an ID for identifying a patient related to medical information managed by an electronic medical record identified by the item "Electronic Medical Record ID". The item "Department ID" is an ID for identifying a department related to medical information managed by an electronic medical record identified by the item "Electronic Medical Record ID". The item "Electronic Medical Record Data" is information related to the file name of the electronic medical record data 2023 related to the electronic medical record identified by the item "Electronic Medical Record ID".

[0102] <3 Example of operation> An example of the operation of the terminal device 10 and the server 20 will be described below.

[0103] Fig. 5 is a flowchart showing an example of the operation of the terminal device 10. Fig. 5 is a flowchart showing an example of the operation when the operator of the terminal device 10 inputs / modifies / adds to the electronic medical record template 2024 by voice input.

[0104] In step S500, a medical professional who is entering data into the electronic medical record template 2024 or the like makes a voice input, preferably including medically designated words, into the speaker 172 of the terminal device 10 that the medical professional operates, and the data processing unit 193 of the terminal device 10 converts the input voice into speech data, which is digital data, and transmits this speech data to the server 20 via the transmission / reception unit 192. Next, the control unit 203 of the server 20 receives the speech data transmitted from the terminal device 10 via the reception control module 2031 and stores it in the memory unit 202. Specifically, for example, the control unit 203 receives the speech data transmitted from the terminal device 10 via the reception control module 2031 and the voice recognition module 2033, and stores it in the memory unit 202 as speech data 2026.

[0105] Next, in step S501, the control unit 203 performs real-time speech recognition using the speech data 2026 accepted in step S500, and causes the display 141 of the terminal device 10 to display the speech recognition result in real time. Specifically, for example, the control unit 203 performs real-time speech recognition using the speech data 2026 accepted in step S500 using the speech recognition module 2033, and causes the presentation control unit 2036 to display the speech recognition result on the display 141 of the terminal device 10. As already mentioned, the speech recognition result in step S501 is data containing a mixture of hiragana, katakana, alphanumeric characters, and punctuation marks, and preferably, blank data is inserted as delimiters between these hiragana, katakana, alphanumeric characters, and punctuation marks. Even if the quality of the real-time conversion of input to kanji conversion or the like is poor, the user can see that input has been made, and can feel at ease. The speech recognition result in step S501 may contain kanji characters among data containing hiragana, katakana, alphanumeric characters, and punctuation marks. What is important about this conversion is that it is a system that prioritizes time over quality.

[0106] Next, in step S502, the control unit 203 terminates the speech recognition process in response to a speech recognition termination instruction input from the medical professional operating the terminal device 10 or a certain period of interruption in speech input using the terminal device 10, and once confirms speech recognition data 2027 that is the result of the speech recognition process. Specifically, for example, the control unit 203 terminates the speech recognition process by the speech recognition module 2033 in response to a speech recognition termination instruction input from the medical professional operating the terminal device 10 or a certain period of interruption in speech input using the terminal device 10, and once confirms speech recognition data 2027 that is the result of the speech recognition process. Then, the speech recognition module 2033 stores the confirmed speech recognition data 2027 in the storage unit 202.

[0107] Next, in step S503, the control unit 203 generates a prompt to be input to the machine learning model 2040 in step S504 (described later) based on the patient's medical history related to the utterance data 2026, the medical record information from the previous visit, the medical treatment summary from the previous hospitalization, and the text of the referral letter. Specifically, for example, the control unit 203 generates a prompt to be input to the machine learning model 2040 in step S504 (described later) based on the patient's medical history related to the utterance data 2026, the medical record information from the previous visit, the medical treatment summary from the previous hospitalization, and the text of the referral letter using the prompt generation module 2034. At this time, the prompt generation module 2034 searches the electronic medical record template 2024 using information (ID) for identifying the patient, and obtains the patient's medical history related to the utterance data 2026, the medical record information from the previous visit, the medical treatment summary from the previous hospitalization, and the text of the referral letter.

[0108] Then, in step S504, the control unit 203 inputs the prompt generated in step S503 and the speech recognition data 2027 confirmed in step S502 to the machine learning model 2040, and obtains kanji-converted speech recognition data 2028 obtained by converting the speech recognition data 2027 from kana to kanji and / or correcting typos, output from the machine learning model 2040. Specifically, for example, the control unit 203 inputs the prompt generated in step S503 and the speech recognition data 2027 confirmed in step S502 to the machine learning model 2040 using the generative model input / output module 2035, and obtains kanji-converted speech recognition data 2028 obtained by converting the speech recognition data 2027 from kana to kanji and / or correcting typos, output from the machine learning model 2040. The generative model input / output module 2035 then stores the obtained kanji-converted speech recognition data 2028 in the storage unit 202.

[0109] Generative AI generally has a somewhat slow output speed, but is characterized by its ability to generate high-quality responses and sentences. Generative AI can also control the output results by adjusting the content of prompts (instructions). This allows generative AI to output highly accurate conversion or completion results that reflect the instructions by specifying in advance the context of the text to be output, the user's usage environment, and specific keywords (e.g., medical terms and proper nouns) through prompts.

[0110] The system according to the present disclosure utilizes the characteristics of generative AI as described above. However, the conversion itself does not necessarily have to utilize generative AI; for example, a general program may be utilized. Examples of general programs include general rule-based algorithms and hybrid programs that combine such algorithms with existing statistical language models. Furthermore, multiple types of general programs may be selected and combined according to the user's requirements (personality, attributes, etc.), the user's usage environment, etc. Regardless of which of these is utilized, the system according to the present disclosure emphasizes conversion that prioritizes "quality" of the two axes of "time" and "quality."

[0111] Specifically, the system according to the present disclosure comprehensively incorporates the following information (a) to (f) and evaluates the quality of generation or conversion.

[0112] (A) User's environmental information User environment information is information about the user's usage environment, such as the operating status of the device used by the user, the network communication status, the network access status, etc. Specifically, for example, in a medical setting, it could be information about the usage environment of the electronic medical record system, information about the medical department to which the user belongs, in a restaurant, information about the usage environment of POS registers, order management systems, etc., and in various companies including IT companies, it could be information about the usage environment of customer relationship management (CRM), inventory management systems, etc.

[0113] (a) Information about the user's intentions The system grasps the final goal and completion requirements of the work, etc. through the work or task that the user is attempting to perform (e.g., entering information into a medical record, adding comments to customer data, creating a report, etc.). In other words, the final goal and completion requirements of the work, etc. grasped from the work, etc. that the user is attempting to perform are an example of information about the user's intention. The system according to the present disclosure then outputs the grasped results.

[0114] (c) Information about the User's customers For example, information about a user's customers includes information about patients in a medical setting, customers in a restaurant, and business partners in various companies. The system disclosed herein performs accurate kanji and alphanumeric conversion by utilizing information about the user's customers, particularly names, ID information, information about preferences and allergies, and purchase history. The system disclosed herein also utilizes a reference dictionary to avoid erroneous conversion of patient names and company names, particularly when automatically converting proper nouns.

[0115] (d) Information about user preferences Information regarding user preferences includes, for example, information regarding the user's writing style and tendency toward spelling variations, whether the user is willing to use abbreviations, etc. The system according to the present disclosure uses such information regarding user preferences to prompt the user to select settings such as whether to correctly expand medical terms and abbreviations or leave company and product names in their original language.

[0116] (e) Information about user habits When a phrase or notation rule specific to the user, or a term that the user is accustomed to using, is input, for example, if a medical professional has a habit of adding the pronunciation of the patient's name after the name when filling out a medical record, such as "(Sato Taro)", the system disclosed herein automatically completes the notation based on that habit.

[0117] (f) Information about the user's next process The system disclosed herein adjusts the layout to match the template format or automatically converts it to a format that can be imported into an existing system by inputting how the generated or converted results will be used (such as transcription into a template, report generation, form output, or integration with other systems).

[0118] By using multifaceted information such as those described above (a) through (f), the system disclosed herein can not only convert words and phrases but also generate high-quality results tailored to the user's various circumstances. Specifically, the system disclosed herein minimizes the user's manual corrections after output, such as converting customer names to accurate kanji characters and replacing alphabetic characters with alphanumeric characters during speech recognition. Furthermore, the system disclosed herein can clearly indicate designated terms in the medical field and keywords related to business processes through prompts, preventing unnecessary or inappropriate language from being mixed in when instructing the generation AI.

[0119] The ultimate goal of the system disclosed herein is to perform conversion that prioritizes quality over time efficiency by utilizing information derived from multiple perspectives, such as the "user's usage environment," "user's intention," "user's customers," "user's preferences," "user's habits," and "user's next process." This increases the accuracy and validity of the generated or converted text, reducing the burden of corrections when the user actually uses the system.

[0120] Meanwhile, in step S550, the control unit 203 calls the electronic medical record template 2024 stored in the memory unit 202. Specifically, for example, the control unit 203 calls the electronic medical record template 2024 stored in the memory unit 202 using the generative model input / output module 2035.

[0121] Next, in step S551, the control unit 203 obtains, from the electronic medical record template 2024 called up in step S550, text containing medical-designated terms linked to this electronic medical record template 2024. Specifically, for example, the control unit 203 uses the prompt generation module 2034 to obtain, from the electronic medical record template 2024 called up in step S550, text containing medical-designated terms linked to this electronic medical record template 2024.

[0122] Next, in step S552, the control unit 203 obtains the medical history of the patient related to the utterance data 2026, the medical record information from the previous visit, the medical treatment summary from the previous hospitalization, and the text of the letter of referral, all of which were obtained in step S503, and the kanji-converted speech-recognition data 2028 obtained in step S504. Specifically, for example, the control unit 203 uses the prompt generation module 2034 to obtain the medical history of the patient related to the utterance data 2026, the medical record information from the previous visit, the medical treatment summary from the previous hospitalization, and the text of the letter of referral, all of which were obtained in step S503.

[0123] Next, in step S553, the control unit 203 inputs the text obtained in step S551 or step S552 as a prompt to the machine learning model 2040, and obtains a document output from the machine learning model 2040. Specifically, for example, the control unit 203 inputs the text obtained in step S551 or step S552 as a prompt to the machine learning model 2040 using the prompt generation module 2034, and obtains a document output from the machine learning model 2040. The machine learning model 2040 used here is a machine learning model 2040 that receives the text obtained in step S551 or step S552 as an input and outputs a document that can be written in the electronic medical record template 2024 based on the input text. Preferably, the output of the machine learning model 2040 includes medically designated terms.

[0124] Next, in step S554, the control unit 203 links the document output from the machine learning model 2040 with the electronic medical record template 2024, using the medical designated terms included in the document output from the machine learning model 2040 in step S553 as a key. Specifically, for example, the control unit 203 uses the medical designated terms included in the document output from the machine learning model 2040 in step S553 as a key, by the electronic medical record data generation module 2037, to link the document output from the machine learning model 2040 with the electronic medical record template 2024. This operation corresponds to the operation of specifying the content to be recorded in the electronic medical record template 2024, using the medical designated terms as a key, based on the speech data 2026, the speech recognition data 2027, and the kanji-converted speech recognition data 2028.

[0125] Then, in step S555, the control unit 203 performs input / modification / overwriting work on the electronic medical record template 2024 based on the record content identified in step S554. Specifically, for example, the control unit 203 causes the electronic medical record data generation module 2037 to perform input / modification / overwriting work on the electronic medical record template 2024 based on the record content identified in step S554.

[0126] 5 can be applied to cases other than when a medical professional inputs / modifies / adds to the electronic medical record template 2024 by voice input. Specifically, for example, the process can be applied to when the owner or employee (user) of a restaurant operates the terminal device 10, or when a person in charge (user) of various companies including an IT company operates the terminal device 10.

[0127] That is, in step S500, an operator such as a restaurant owner or employee, or a person in charge of various companies, inputs voice into the speaker 172 of the terminal device 10 that he or she operates. After this process, the various processes up to S502 are the same as those in the case where the operator is a medical professional as described above.

[0128] Next, in step S503, the control unit 203 generates a prompt to be input to the machine learning model 2040 in step S504 based on the speech data 2026 of the user's customer and at least one of the user's environmental information, information about the user's customer, and a document before the voice recognition data was acquired. Here, "customers" include, for example, patients in a medical setting, customers at a restaurant, and business partners at various companies. In addition, the information about the user's customer includes, for example, a document before the voice recognition data 2027 was acquired.

[0129] For example, the information about the user's customers handled in each of the aforementioned industries / sectors is as follows:

[0130] (1) Medical field (customers: patients) For example, this may include hospital visit history, treatment history, the contents of the medical record from the previous visit (document before the voice recognition data 2027 is acquired), the contents of the medical treatment summary from the previous hospitalization (document before the voice recognition data 2027 is acquired), and the contents of the referral letter. In particular, it is important to accurately convert the patient's name into kanji and to handle the identification information (patient ID) for matching with the hospitalization history in alphanumeric characters. Here, "the previous" specifically refers to the visit / hospitalization before the voice recognition data based on the patient's speech data 2026 is acquired.

[0131] (2) Restaurants (customers) Examples of such information include store visit history, order history, information about allergies or preferences, review comments, the contents of questionnaires completed by customers (documents prior to acquisition of voice recognition data 2027), and the contents of testimonials from other stores. In particular, it is required to accurately convert the names of customers (which may be pronounced the same but written in different kanji characters) without error using voice recognition. Here, the questionnaires completed by customers are specifically those obtained prior to acquisition of voice recognition data based on the customer's speech data 2026.

[0132] (3) Various companies (customers: business partners) Examples of such information include information about business negotiation history, information about product / service implementation history, information about support history, product inquiry logs, the contents of completed questionnaires (documents prior to the acquisition of speech recognition data 2027) by business partners, and text of testimonials from other companies. In particular, since corporate names and names of personnel may be written in alphanumeric characters or katakana / romanized characters, character conversion or spelling correction is required at the speech recognition stage. Here, the completed questionnaires by business partners are specifically those obtained prior to the acquisition of speech recognition data based on the speech data 2026 of business partners.

[0133] Specifically, for example, the prompt generation module 2034 generates a prompt to be input to the machine learning model 2040 in step S504 based on the user's customer utterance data 2026 and at least one of the customer's usage history, a summary of interactions with the customer, the contents of various reports from previous usage, and the introductory text. At this time, the prompt generation module 2034 acquires various pieces of customer information, for example, using information for identifying the customer (customer ID). Furthermore, for example, the prompt generation module 2034 may take into account user environment information when generating a prompt. Furthermore, the utterance data 2026, the customer's usage history, a summary of interactions with the customer, the contents of various reports from previous usage, the introductory text, and the user's environment information do not have to be input to the machine learning model 2040 in a manner to be incorporated into the prompt, but may instead be input as explanatory variables of the machine learning model 2040, for example.

[0134] For example, in a medical setting, the prompt generation module 2034 searches the electronic medical record template 2024 to obtain a patient's profile and history, medical record information from the previous visit, information about the medical treatment summary from the previous visit, the text of the referral letter, etc. In a restaurant, for example, the prompt generation module 2034 searches a customer database (not shown) to obtain order history, review history, text of testimonials from other stores, etc. Furthermore, in various businesses, the prompt generation module 2034 searches a customer information database (not shown) to obtain the names of business partners in the profile, the names and departments of the people in charge, business negotiation history, inquiry logs, text of testimonials from other companies, etc.

[0135] Furthermore, in the speech recognition process, accurate recognition and conversion of proper nouns of customers (patients, customers, business partners, etc.) is particularly important. For example, if a patient's name is spoken as "Sato," and candidate text such as "Sato" or "sato" is generated, the prompt generation module 2034 estimates the correct kanji conversion, "Sato," and associates it with alphanumeric characters such as the patient ID "ID-12345" as necessary. On the other hand, if a company name is spoken in Roman letters or katakana, the speech recognition system must accurately convert the name or organization name. To avoid misconversions and spelling variations, it is recommended to utilize dictionary information and template information referenced by the machine learning model 2040.

[0136] In this way, by generating prompts based on the speech data 2026 and information about the user's customer (including patient information), it is possible to optimize the dialogue and proposal content (i.e., kanji-converted speech recognition data 2028) output from the machine learning model 2040 to suit the situation and attributes of the customer. In addition, by accurately converting and managing unique information such as customer name and customer ID into text, there are advantages in that corrections are reduced, the accuracy of history matching and the accuracy of acquiring additional information are improved, and matching errors and acquisition errors can be reduced.

[0137] Next, in step S504, the control unit 203 may acquire kanji-converted speech recognition data 2028 without inputting the prompt generated in step S503 and the speech recognition data 2027 confirmed in step S502 into the machine learning model 2040.

[0138] Specifically, for example, the generative model input / output module 2035 may acquire the kanji-converted speech recognition data 2028 by analyzing, with a rule-based algorithm, a combination of at least two or more of the speech data 2026, the speech recognition data 2027, a customer's usage history (information about the user's customer), a summary of interactions with the customer (information about the user's customer), the contents of various reports from the previous use (documents before the speech recognition data 2027 was acquired), the text of an introductory comment, or information about the user's environment, etc. Also, for example, the generative model input / output module 2035 may acquire the kanji-converted speech recognition data 2028 by using both the machine learning model 2040 and a rule-based algorithm.

[0139] The rule-based algorithm used to acquire the kanji-converted speech recognition data 2028 may be, for example, the following known algorithm (a) or (b).

[0140] (a) Algorithm for generating continuous phoneme data First, the speech signal is divided into frames at short intervals, and speech features of each frame are calculated. Examples of speech features include frequency spectrum, formant frequency, and zero-crossing rate. Next, the calculated speech features are analyzed, and phoneme candidates are estimated by applying thresholds or rules based on the features. Next, continuous data of the estimated phonemes is generated. Note that the above-mentioned frame division of the speech signal and extraction of speech features often use common signal processing techniques (e.g., filtering and Fourier transform).

[0141] (b) an algorithm that converts a sequence of phonemes into a word or sentence First, the phoneme sequence is compared with a predefined word dictionary to extract matching word candidates. Next, grammar rules or syntactic analysis are applied to score the candidate word sequence. Examples of grammar rules and syntactic analysis include subject-verb-object sentence structures and phoneme connection rules. Next, based on the scoring results, the most likely word sequence or sentence is determined. Here, a state transition model (e.g., a finite state machine) or regular expressions is often used to apply the grammar rules. Furthermore, heuristic methods based on connection probability or confidence are often introduced for dictionary lookup and scoring.

[0142] In this way, speech recognition processing using rule-based algorithms is a process that is executed by combining speech features and grammar rules, and is useful as a method for realizing speech recognition without using a machine learning model 2040.

[0143] To summarize the above, the control unit 203 acquires the kanji-converted voice recognition data 2028 by analyzing a combination of at least two of (i) speech data 2026, (ii) user environmental information, (iii) documents before acquiring the voice recognition data, (iv) information about the user's customers, or (v) voice recognition data 2027.

[0144] 6 is a flowchart showing an example of the operation of the server 20. FIG. 6 is a flowchart showing an example of the operation when the operator of the terminal device 10 generates document data using the electronic medical record template 2024 stored in the server 20.

[0145] In step S600, the control unit 203 calls the electronic medical record template 2024 stored in the memory unit 202. Specifically, for example, the control unit 203 calls the electronic medical record template 2024 stored in the memory unit 202 using the generative model input / output module 2035.

[0146] Next, in step S601, the control unit 203 obtains, from the electronic medical record template 2024 called in step S600, data that was previously input to this electronic medical record template 2024. Specifically, for example, the control unit 203 uses the generative model input / output module 2035 to obtain, from the electronic medical record template 2024 called in step S600, data that was previously input to this electronic medical record template 2024.

[0147] Meanwhile, in step S602, the control unit 203 generates, from the electronic medical record template 2024 called up in step S600, a prompt including a medical designated term linked to this electronic medical record template 2024. Specifically, for example, the control unit 203 causes the prompt generation module 2034 to generate, from the electronic medical record template 2024 called up in step S600, a prompt including a medical designated term linked to this electronic medical record template 2024.

[0148] Next, in step S603, the control unit 203 inputs the prompt generated in step S602 to the machine learning model 2040, and accepts output of a document merge template 2025 with a form input function based on the input prompt from the machine learning model 2040. Specifically, for example, the control unit 203 inputs the prompt generated in step S602 to the machine learning model 2040 using the generation model input / output module 2035, and accepts output of a document merge template 2025 with a form input function based on the input prompt from the machine learning model 2040. The document template 2025 output by the machine learning model 2040 in step S603 is, as described above, a document template 2025 that includes, as a character string that is unlikely to collide, the medical-designated term included in the prompt generated in step S602. In addition, a mail merge template 2025 with form input function is a document template 2025 into which mail merge fields and mail merge forms (a UI with form functions that allow easy selection using select boxes, check boxes, free text fields, pull-down menus, etc.) are inserted.

[0149] In step S604, the control unit 203 selects the merged document template 2025 generated in step S603. Specifically, for example, the control unit 203 causes the presentation control unit 2036 to select the merged document template 2025 generated in step S603.

[0150] When selecting a mail merge document, a special display such as faint text around the cursor indicates that it is an input candidate, and mail merge template candidates can also be displayed. If the template is inappropriate, it will disappear when you start writing, or you will be prompted to change to another template. A template candidate may be selected from the contents of the medical record immediately before that, and a special UI may be displayed indicating that it is an input candidate as described above.

[0151] Next, in step S605, the control unit 203 creates document data based on the merged document template 2025 selected in step S604. Specifically, for example, the control unit 203 creates document data using the generative model input / output module 2035 based on the merged document template 2025 selected in step S604.

[0152] Next, in step S606, the control unit 203 highlights the inserted portion in the document data created in step S605. Specifically, for example, the control unit 203 causes the presentation control unit 2036 to display the document data created in step S605 on the display 141 of the terminal device 10, and at this time, highlights the inserted portion so that it can be distinguished from the other portions.

[0153] Next, in step S607, the control unit 203 inputs a form into the document data generated in step S605. Specifically, for example, the control unit 203 acquires the content input into the form by the medical worker via the terminal device 10 using the presentation control unit 2036, and displays the input content on the display 141 of the terminal device 10.

[0154] Next, in step S608, the control unit 203 confirms the content input into the form in step S607. Specifically, for example, the control unit 203 accepts, from the medical professional operating the terminal device 10, an instruction to confirm the form input content displayed on the display 141 of the terminal device 10 via the presentation control unit 2036, and confirms the form input content.

[0155] Then, in step S609, the control unit 203 inputs the document data confirmed in step S608 into the machine learning model 2040, and causes the machine learning model 2040 to output document data with improved sentence fluency. Specifically, for example, the control unit 203 inputs the document data confirmed in step S608 into the machine learning model 2040 using the generative model input / output module 2035, and causes the machine learning model 2040 to output document data with improved sentence fluency. The machine learning model 2040 here may be a general large-scale language model that has not undergone any special training.

[0156] <4 Screen example> Hereinafter, examples of screens output to the terminal device 10 will be described with reference to FIGS.

[0157] FIG. 7 is a diagram showing an example of a screen on the display 141 of the terminal device 10, in which the progress and results of the speech recognition process for the speech input made by the operator of the terminal device 10 are displayed.

[0158] A screen 700 of the display 141 of the terminal device 10 is provided with an area 701 for displaying the progress and results of speech recognition processing of a speech input made by the operator of the terminal device 10, and this area 701 displays a speech recognition processing result 702 by the speech recognition module 2033 and the machine learning model 2040. In the example shown at the top of the figure, as already described, the speech recognition module 2033 outputs speech recognition data 2027, which is the speech recognition processing result, as data containing a mixture of hiragana, katakana, alphanumeric characters, and punctuation marks, and preferably with blank data inserted as delimiters between the hiragana, katakana, alphanumeric characters, and punctuation marks. Therefore, the speech recognition processing result 702 at the top of the figure is data containing a mixture of hiragana, katakana, alphanumeric characters, and punctuation marks, and further with blank data inserted as delimiters between the hiragana, katakana, alphanumeric characters, and punctuation marks. The speech recognition processing result 702 is performed in real time and is also displayed on the display 141 in real time.

[0159] The prompt generation module 2034 obtains patient data related to the voice input from the electronic medical record template 2024 and generates a prompt using this data. As an example, the prompt generation module 2034 obtains data such as that displayed in area 703 in the drawing from past medical record data, and the generation model input / output module 2035 inputs this data as a prompt to the machine learning model 2040.

[0160] The speech recognition processing result 702 at the bottom of the figure is kanji-converted speech recognition data 2028, which is output from the machine learning model 2040. It can be seen that accurate kana-kanji conversion has been performed in this kanji-converted speech recognition data 2028.

[0161] FIG. 8 is a diagram for explaining the procedure for generating a referral letter to another hospital, which is document data, starting from a previous natural-language referral letter stored in the electronic medical record template 2024.

[0162] First, the prompt generation module 2034 searches the electronic medical record template 2024 and acquires medical information suitable for creating document data from the medical information contained in the electronic medical record template 2024. In the illustrated example, it is a referral letter (natural language) from the previous medical site.

[0163] Next, the prompt generation module 2034 designs a prompt that the machine learning model 2040 uses to generate a document template 2025 in which the item names and values ​​of medically designated terms, etc. are converted into a form separated by particles, symbols, etc., and then the generation model input / output module 2035 inputs the prompt designed and output by the prompt generation module 2034 into the machine learning model 2040, causing this machine learning model 2040 to output a document template 2025 in which the item names and values ​​of medically designated terms, etc. are converted into a form separated by particles, symbols, etc.

[0164] The document template 2025 is a natural text structured into table information by recognizing symbols such as medically designated terms and particles. Alternatively, if the output from the machine learning model 2040 is natural text, the presentation control unit 2036 converts the natural text into a structured table information by recognizing symbols such as medically designated terms and particles. When a medical professional operating the terminal device 10 generates a letter of referral, a table-structured document template 2025 as shown in Figure 8 is displayed on the display 141 of the terminal device 10, and the medical professional selects and inputs the items to be output in the letter of referral.

[0165] Meanwhile, the prompt generation module 2034 generates a prompt that includes medical designated terms and causes the machine learning model 2040 to output a desired document template 2025. This prompt includes the medical designated terms as symbols that are unlikely to clash. The generation model input / output module 2035 inputs the prompt generated by the prompt generation module 2034 to the machine learning model 2040, and causes the machine learning model 2040 to output a merged document template 2025 that includes the medical designated terms as symbols that are unlikely to clash.

[0166] Thereafter, the generation model input / output module 2035 generates a referral letter, which is document data, by inserting the input contents of the electronic medical record template 2024 using the medically designated terms as a key.

[0167] FIG. 9 is a diagram showing an example of a text expression corresponding to the merge template field function and a text expression corresponding to the merge form function in the merge document template 2025. In FIG.

[0168] Figure 10 is a diagram showing an example of a prompt generated by the prompt generation module 2034 when the generation model input / output module 2035 inputs the speech recognition data 2027 output from the speech recognition module 2033 into the machine learning model 2040 to obtain kanji-converted speech recognition data 2028.

[0169] 11 is a diagram showing an example of a screen displayed on the display 141 of the terminal device 10. In the figure, the content to be reflected in the electronic medical record template 2024, i.e., the content to be input, etc. (on the left side of the figure) and the voice recognition processing results (on the right side of the figure) are displayed. A screen for displaying kanji conversion candidates based on the voice input is superimposed on the screen showing the voice recognition processing results.

[0170] FIG. 12 is a diagram showing an example of a screen displayed on the display 141 of the terminal device 10 when the operator of the terminal device 10 performs input work on the electronic medical record template 2024.

[0171] On the left side of the screen 1200 of the display 141 of the terminal device 10, a screen 1201 is displayed showing input contents identified as a result of a voice recognition process based on the input items of the electronic medical record template 2024 stored in the memory unit 202 of the server 20 and the utterance data 2026 from the operator, and on the upper right side of the screen 1200 of the display 141, a voice input guidance screen 1202 is displayed, and on the lower right side of the screen 1200, a screen 1203 showing the voice input results based on the utterance data 2026 is displayed. Furthermore, on the lower side of the screen 1200, buttons 1205 and 1206 are displayed for instructing the start of recording the utterance data 2026 and the saving of the utterance data 2026.

[0172] The operator of the terminal device 10 performs an operation input such as clicking on these buttons 1205 and 1206 using the input device 13 to issue an instruction to start recording the speech data 2026 or an instruction to save the speech data 2026 .

[0173] Fig. 13 is a diagram showing details of the screen 1201 shown in Fig. 12. As already explained, the screen 1300 (1201) is a screen showing input contents identified as a result of the voice recognition process based on the input items of the electronic medical record template 2024 and the speech data 2026 from the operator.

[0174] A screen 1300 displayed on the display 141 of the terminal device 10 displays input items 1301 of the electronic medical record template 2024 and input content 1302 associated with these input items 1301. The input items 1301 also display a numeric string 1303 for identifying these input items 1301. As the voice recognition module 2033 performs voice recognition processing, voice input results are sequentially entered into the input content 1302. In addition, some items have already been entered in the input content 1302 of the electronic medical record template 2024, and the voice input is being performed to add to or correct the already entered items. As shown in FIG. 13 , the already entered items are displayed in a specific color as "no corrections," and items that have been added to or corrected by voice input are displayed in a color different from the already entered items.

[0175] Fig. 14 is a diagram showing details of the screen 1202 shown in Fig. 12. As already explained, the screen 1400 (1202) is a guidance screen for voice input.

[0176] A screen 1400 of the display 141 of the terminal device 10 displays input items 1401 that are input items of the electronic medical record template 2024 and include medically designated terms, and examples 1402 of input content to be entered for these input items 1401. Furthermore, if there are items already entered in the electronic medical record template 2024, the already entered items are displayed in the location of the examples 1402.

[0177] FIG. 15 is a diagram showing a screen that displays examples of kanji conversion candidates that are displayed as a pop-up on the screen 1200 shown in FIG. 12 when the operator of the terminal device 10 is performing voice input using the screen 1200. In FIG.

[0178] A screen 1500 of the display 141 of the terminal device 10 displays hiragana notation 1501 and conversion candidate examples 1502 when the voice recognition module 2033 performs kanji conversion based on the speech data 2026. The operator of the terminal device 10 confirms the kanji conversion by selecting one of the conversion candidate examples 1502 using the input device 13.

[0179] Figure 16 is a diagram showing an example of electronic medical record template data provided by another medical institution when importing an electronic medical record template. The structure of the electronic medical record template data is the same as that shown in Figure 13, so a detailed explanation will be omitted. As it is electronic medical record template data, it even includes profile information.

[0180] FIG. 17 is a diagram for explaining the procedure for generating a letter of introduction text that is generated by the server 20 based on electronic medical record template data.

[0181] A list of electronic medical record template data (input contents) for a specific patient is displayed in the upper part of Fig. 17. From this list of input contents, the operator of the server 20 selects input contents necessary for generating the referral letter text and input contents not necessary (necessary / not necessary for the script).

[0182] Once the selection of the input content is complete, the server 20 generates a script to be input into the sentence generation task of the large-scale language model (machine learning model 2040). The generated script is displayed in the middle of Fig. 17. As already explained, in this case, some numerical values ​​(in the illustrated example, the numerical values ​​of the test results indicating the blood sampling results) are replaced with dummy values ​​when generating sentences through the sentence generation task to check whether the sentences are generated accurately.

[0183] The results of inputting the script into a sentence generation task of a large-scale language model are shown in the bottom panel of Figure 17.

[0184] FIG. 18 is a diagram showing an example of character string replacement performed by the voice recognition module 2033 and the machine learning model 2040 in the procedure for generating the letter of introduction text shown in FIG.

[0185] <5. Effects of one embodiment> As described above in detail, the system 1 of this embodiment allows the recorded content of medical procedures, such as an input electronic medical record, to be added to or corrected through a simple procedure. This point will be described in detail below.

[0186] The medical industry is one in which mistakes cannot be tolerated. Therefore, there is a high demand for template-based structured data in the medical industry. Structured data can reduce medical errors. Electronic medical record templates can standardize business processes.

[0187] However, inputting structured data into electronic medical records is extremely time-consuming. For example, at a certain medical facility's admission and discharge support center, there was approximately six pages of input content, and it took about 20 minutes per patient to instruct which fields to input the content into and enter it there. With the system 1 according to this embodiment, this work could be reduced to about five minutes.

[0188] There are three reasons why using voice recognition to input structured data has not been used in the field until now: first, the accuracy of voice recognition is not sufficient, second, there are easier ways to input data than voice recognition, and third, it is difficult to correct mistakes.

[0189] In the system 1 according to the present embodiment, the first problem, the problem of speech recognition accuracy, is solved by limiting the usage scenarios, simultaneously displaying the template input item names and input content candidates for these input items, and narrowing down the input speech patterns. In the system 1 according to the present disclosure, the WER (Word Error Rate) associated with speech recognition is reduced from 6% to about 2%.

[0190] The second problem was solved by having multiple choice questions, which is an easier input method than voice recognition, be used before voice recognition, and then displaying the results. Any parts of the input that were insufficient could be corrected or added using voice recognition.

[0191] The third problem, that of making it difficult to correct mistakes, is solved by displaying other candidates after inputting the text, allowing the user to select from the other candidates, and by making it easy to check what voice was used to input the text.

[0192] In particular, in the system 1 of this embodiment, since the input content in multiple-choice format usually contains medically designated terms, speech recognition processing is performed using these medically designated terms as keys to identify at least the input content in multiple-choice format, thereby further improving the accuracy of speech recognition.

[0193] <6 Variations> It should be noted that the above-described embodiments have been described in detail to clearly explain the present disclosure, and are not necessarily limited to those including all of the described configurations. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0194] As an example, in the system 1 of the embodiment described above, the machine learning model 2040 is located in the server 20, but it may be located in, for example, the external server 40 shown in FIG.

[0195] Furthermore, in the above-described embodiment of system 1, accurate kana-kanji conversion of spoken data was performed using speech recognition processing and a machine learning model for input / append / overwrite operations into the electronic medical record template 2024, but the target of accurate kana-kanji conversion of spoken data using speech recognition processing and a machine learning model is not limited to the contents of the electronic medical record template 2024, and even accurate kana-kanji conversion may be performed using a machine learning model on text data input by a user using the input device 13 of the terminal device 10, etc., without going through speech recognition processing.

[0196] Furthermore, in the system 1 of the embodiment described above, the user operating the terminal device 10 inputs and instructs the timing of inputting and outputting speech data, but the server 20 or the like may make an autonomous decision based on at least one of the punctuation marks, line break codes, number of characters in the text data input by the user, and a predetermined period of interruption of the user's input work.

[0197] Figure 19 shows a flowchart of the input content specification operation of the electronic medical record template 2024, which starts with text input. When text is entered from the keyboard, "text indicating that the input has been correct" is displayed in real time. After that, the end of text input is determined based on some signal, such as the input of punctuation, a line break, or no input for several seconds, and the "text entered from the keyboard" is input into the generation AI prompt, where typos are corrected and kanji conversion is performed. Furthermore, by adding electronic medical record information and peripheral information to the prompt at this time, accuracy can be improved.

[0198] The operations shown in the flowchart of FIG. 19 are substantially the same as those shown in the flowchart of FIG. 5, and therefore a description of the substantially identical parts will be omitted.

[0199] In step S1900, the user of the terminal device 10 operates the keyboard 131 to input text data via the input device 13. The operations of the following steps S1901 to S1955 are the same as steps S501 to S555 in FIG.

[0200] In step S1903, the control unit 203 generates a prompt to be input to the machine learning model 2040 based on the text data input from the input device 13. Specifically, for example, the control unit 203 corrects typos or performs appropriate kanji conversion including medical terminology on the text data using the prompt generation module 2034. As a result, the control unit 203 corrects the text data with high accuracy and generates a prompt to be input to the machine learning model 2040. Furthermore, the control unit 203 causes the prompt generation module 2034 to reflect, as necessary, patient information, a disease name list, dictionary data containing medical terminology, a medical terminology list, and the like contained in the electronic medical record template 2024 in the content of the prompt. This allows for more accurate conversion results to be obtained.

[0201] Then, in step S1904, the control unit 203 displays the kanji-converted speech recognition data 2028 acquired from the machine learning model 2040 on a display device (not shown) provided in the server 20. This allows the user to not only check the conversion result (kanji-converted speech recognition data 2028) through the display device, but also manually edit the conversion result as necessary. Meanwhile, the control unit 203 confirms the conversion result by receiving information that there is no problem with the conversion result, and reflects the confirmed conversion result in the electronic medical record template 2024.

[0202] As a result, even with simple text input from the keyboard 131, punctuation, line breaks, interruptions in input, etc. are used as triggers to automatically correct typos and convert kanji characters using the generation AI (machine learning model 2040). Furthermore, by reflecting in the prompts patient information and disease name lists already stored in the electronic medical record (collectively referred to as "electronic medical record information"), as well as dictionary data containing medical terms and medical terminology lists (collectively referred to as "peripheral information"), more highly accurate conversion that is more in line with the context can be achieved. This reduces input errors and spelling variations in medical settings, improving the reliability and creation efficiency of medical records.

[0203] The information to be input to the machine learning model 2040 may also include the department to which the medical professional who owns the terminal device 10 belongs. The machine learning model 2040 may also be trained using information related to the department.

[0204] Furthermore, the content of the prompt generated by the prompt generation module 2034 of the above-described embodiment may be input to the machine learning model 2040 by the generation model input / output module 2035, rather than as a prompt. If the machine learning model 2040 is, for example, a large-scale language model, it is possible to obtain an output close to the desired result without using a prompt. However, if the desired result is not output, it is preferable to repeatedly input the same information to the machine learning model 2040 and gradually obtain an output closer to the desired result.

[0205] Furthermore, if a medical professional operating the terminal device 10 opens the electronic medical record template 2024 and views the medical information of the patient related to the utterance data 2026 prior to the speech recognition processing by the speech recognition module 2033, the prompt generation module 2034 monitors which patient's electronic medical record template 2024 is being viewed on each terminal device 10, and when the speech recognition processing by the speech recognition module 2033 starts, it can search the electronic medical record template 2024 using information to identify the patient and obtain the patient's medical information in advance. This patient's medical information is used for the speech recognition processing by the machine learning model 2040.

[0206] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0207] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, and Java (registered trademark).

[0208] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

[0209] <7 Notes> The matters described in the above embodiments will be supplemented below.

[0210] (Appendix 1) A program for operating a computer having a processor and a memory, wherein the memory stores a machine learning model that receives input speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data by performing kana-to-kanji conversion on the speech recognition data; the program causes the processor to execute the following steps: a first step of accepting information input from a user; a second step of presenting the information entered in the first step to the user; a third step of determining when the user has finished entering the information accepted in the first step; a fourth step of inputting the information accepted in the first step into the machine learning model until the end of input is determined in the third step, as well as inputting environmental information and information into the machine learning model, and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0211] (Appendix 2) A program for operating a computer having a processor and a memory, the memory storing a machine learning model that receives input of speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data by performing kana-to-kanji conversion on the speech recognition data, the program causing the processor to execute the following steps: a first step of accepting input of information about the user's customers from a user; a second step of presenting the information entered in the first step to the user; a sixth step of determining whether the user has finished entering the information accepted in the first step based on at least one of the punctuation marks included in the information, line feed codes, the number of characters in the information, and a predetermined period of interruption in the user's input; a fourth step of inputting the information accepted in the first step into the machine learning model until the end of input is determined in the sixth step, and inputting environmental information and information about the customer into the machine learning model as prompts, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0212] (Appendix 3) A program for operating a computer having a processor and a memory, the memory storing a machine learning model that receives speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input, and outputs kanji-converted speech recognition data by performing kana-kanji conversion on the speech recognition data, the program causing the processor to perform the following steps: a seventh step of accepting input of utterance data relating to the user's customer from a user; an eighth step of performing speech recognition on the utterance data accepted in the seventh step to obtain speech recognition data; a ninth step of presenting the speech recognition data acquired in the eighth step to the user; and a ninth step of displaying the utterance data accepted in the seventh step. a 10th step of determining whether a user has finished inputting speech data based on at least one of punctuation marks, line break codes, the number of characters of information, and a predetermined period of interruption in the user's input, which are included in the speech recognition data; an 11th step of inputting the speech recognition data acquired in the 8th step into a machine learning model until the end of input is determined in the 10th step, inputting environmental information and customer information into the machine learning model as prompts, and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a 12th step of presenting the kanji-converted speech recognition data output in the 11th step to the user.

[0213] (Appendix 4) 4. The program of claim 2 or 3, wherein the user is a healthcare professional and the customer is a patient for whom the user provides medical care.

[0214] (Appendix 5) The user is a medical professional, and the environmental information is information relating to the medical department to which the user belongs.

[0215] (Appendix 6) 5. The program of claim 4, wherein the information about the user's customer is medical record information and / or medical summary information from the customer's most recent visit.

[0216] (Appendix 7) A program for operating a computer having a processor and a memory, wherein the memory stores a machine learning model that receives input speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data by performing kana-to-kanji conversion on the speech recognition data, and the program causes the processor to execute a thirteenth step of accepting input of utterance data from a user, a fourteenth step of performing speech recognition on the utterance data accepted in the thirteenth step to obtain speech recognition data, a fifteenth step of inputting the speech recognition data obtained in the fourteenth step into the machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and a sixteenth step of presenting the kanji-converted speech recognition data, which is the output of the fifteenth step, to the user.

[0217] (Appendix 8) The memory stores structured data of an electronic medical record template, and the structured data is data in which input fields of the electronic medical record template are associated with the input content, and the input content is linked to information for identifying the patient. In a 15th step, the program described in Appendix 7 searches the electronic medical record template using the information for identifying the patient to obtain the input content, which is the input content when the patient most recently visited or was hospitalized, and inputs the voice recognition data into a machine learning model, and also inputs the input content when the patient most recently visited or was hospitalized into the machine learning model as a prompt.

[0218] (Appendix 9) The memory stores structured data of an electronic medical record template, the structured data being data in which input fields of the electronic medical record template are associated with the input content, and the input content is linked to information for identifying the patient; the program further causes the processor to execute a 24th step prior to the 13th step of accepting viewing of the structured data of the electronic medical record template linked to the patient related to the speech data; in the 13th step, the electronic medical record template is searched using information for identifying the patient who has accepted viewing of the structured data of the electronic medical record template to obtain the input content, which is the input content when the patient most recently visited or was hospitalized; and in a 15th step, the speech recognition data is input into a machine learning model, and the input content when the patient most recently visited or was hospitalized is input into the machine learning model as a prompt.

[0219] (Appendix 10) A program as described in Appendix 7, in which structured data of an electronic medical record template is stored in the memory, the structured data being data in which input fields of the electronic medical record template and input content are associated, the input fields including medical designated terms, and the speech data including medical designated terms, and the program further causes the processor to execute a 17th step of identifying the content to be recorded in the electronic medical record template based on the medical designated terms included in the speech data.

[0220] (Appendix 11) 11. The program of claim 10, wherein the speech data is for a specific patient, and in a 15th step, the speech recognition data is input into the machine learning model and an electronic medical record template for the specific patient is input into the machine learning model as a prompt.

[0221] (Appendix 12) 8. The program of claim 7, wherein the speech data is for a specific patient and the memory stores a medical history of the specific patient, and in a fifteenth step, the speech recognition data is input into a machine learning model and the medical history of the specific patient is input into the machine learning model as a prompt.

[0222] (Appendix 13) The program further causes the processor to execute an 18th step of presenting to the user an electronic medical record template including the record content identified in the 17th step, and further, in the 18th step, changes the presentation manner between the identified record content and the electronic medical record template other than the record content and presents them.

[0223] (Appendix 14) In the structured data of the electronic medical record template, the input content includes multiple-choice input content for selecting an option, and in step 17, the program described in Appendix 10 identifies the recording content to be recorded in the electronic medical record template based on at least the multiple-choice input content of the input content of the electronic medical record template.

[0224] (Appendix 15) The program according to claim 7, wherein in step 14, the speech recognition data has blank data inserted between each hiragana, katakana, alphanumeric character, and punctuation mark.

[0225] (Appendix 16) a computer program for operating a computer having a processor and a memory, the memory storing structured data of an electronic medical record template, the structured data being data in which input fields of the electronic medical record template are associated with input contents, the input fields including medical-designated terms; and the memory storing a machine learning model that receives the medical-designated terms as input and outputs a document template including the medical-designated terms as character strings that are unlikely to clash; the program causing the processor to execute a 19th step of acquiring the structured data of the electronic medical record template from the memory; a 20th step of inputting the medical-designated terms included in the structured data of the electronic medical record template acquired in the 19th step into the machine learning model and outputting a document template including the medical-designated terms as character strings that are unlikely to clash; a 21st step of acquiring structured data of an electronic medical record template relating to a specific patient from the structured data of the electronic medical record template; and a 22nd step of inserting the structured data into the document template output in the 20th step using the medical-designated terms included in the structured data of the electronic medical record template relating to the specific patient as a key, to generate document data.

[0226] (Appendix 17) The program further causes the processor to perform a 23rd step of presenting the document data generated in the 22nd step to a user. 17. The program according to claim 16,

[0227] (Appendix 18) A program according to appendix 17, in which in a 23rd step, the structured data inserted into the document template and document data other than the structured data inserted into the document template are presented in different presentation modes.

[0228] (Appendix 19) A program according to appendix 18, wherein in a 23rd step, input items of the structured data inserted into the document template and input contents corresponding to these input items are presented in pairs.

[0229] (Appendix 20) A program as described in Appendix 19, in which in step 23, a selection input of an input item and its corresponding input content to be inserted into the document data from among the input items of structured data inserted into the document template and the input content corresponding to the input items, which are presented in pairs, is accepted, and the program further causes the processor to execute step 24 of inserting into the document data the input item for which the selection input was accepted and the input content corresponding to the input item.

[0230] (Appendix 21) 17. The program of claim 16, wherein the document templates include at least one of a medical summary, a referral letter, and a pharmaceutical company report template.

[0231] (Appendix 22) An information processing device comprising a processor and a memory, wherein the memory stores a machine learning model that receives input of speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data that has been subjected to kana-kanji conversion of the speech recognition data; the processor executes the following steps: a first step of accepting information input from a user; a second step of presenting the information entered in the first step to the user; a third step of determining when the user has finished entering the information accepted in the first step; a fourth step of inputting the information accepted in the first step into the machine learning model until the end of input is determined in the third step, and also inputting environmental information and information into the machine learning model, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data that is the output of the fourth step to the user.

[0232] (Appendix 23) an information processing device having a processor and a memory, the memory storing a machine learning model that receives input of speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data that has been subjected to kana-kanji conversion of the speech recognition data; the processor executes the following steps: a first step of accepting input of information about the user's customers from a user; a second step of presenting the information entered in the first step to the user; a sixth step of determining whether the user has finished entering the information accepted in the first step based on at least one of the punctuation marks included in the information, a line feed code, the number of characters in the information, and a predetermined period of interruption in the user's input; a fourth step of inputting the information accepted in the first step into the machine learning model until the end of input is determined in the sixth step, and inputting environmental information and information about the customer into the machine learning model as prompts, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data that is the output of the fourth step to the user.

[0233] (Appendix 24) An information processing device having a processor and a memory, wherein the memory stores a machine learning model that receives speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input, and outputs kanji-converted speech recognition data by performing kana-kanji conversion on the speech recognition data, and the processor performs a seventh step of receiving, from a user, input of speech data relating to the user's customer, an eighth step of performing speech recognition on the speech data received in the seventh step to obtain speech recognition data, a ninth step of presenting the speech recognition data obtained in the eighth step to the user, and a ninth step of providing the user with the speech recognition data received in the seventh step. an information processing device that executes the following steps: a tenth step of determining the end of input based on at least one of punctuation marks, line break codes, the number of characters of the information, and a predetermined period of interruption of input by the user, which is included in the voice recognition data; an eleventh step of inputting the voice recognition data acquired in the eighth step into a machine learning model until the end of input is determined in the tenth step, and inputting environmental information and information about the customer into the machine learning model as prompts, and causing the machine learning model to output kanji-converted voice recognition data corresponding to the input information; and a twelfth step of presenting the kanji-converted voice recognition data that is the output of the eleventh step to the user.

[0234] (Appendix 25) An information processing device comprising a processor and a memory, wherein the memory stores a machine learning model that receives input speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data that has been subjected to kana-to-kanji conversion of the speech recognition data; the processor executes a thirteenth step of accepting input of utterance data from a user, a fourteenth step of performing speech recognition on the utterance data accepted in the thirteenth step to obtain speech recognition data, a fifteenth step of inputting the speech recognition data obtained in the fourteenth step to the machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and a sixteenth step of presenting the kanji-converted speech recognition data that is the output of the fifteenth step to the user.

[0235] (Appendix 26) an information processing device having a processor and a memory, wherein the memory stores structured data of an electronic medical record template, the structured data being data in which input fields of the electronic medical record template are associated with input contents, the input fields including medical-designated terms; and the memory stores a machine learning model that receives the medical-designated terms as input and outputs a document template including the medical-designated terms as character strings that are unlikely to clash; the processor executes a 19th step of acquiring the structured data of the electronic medical record template from the memory; a 20th step of inputting the medical-designated terms included in the structured data of the electronic medical record template acquired in the 19th step into the machine learning model and outputting a document template including the medical-designated terms as character strings that are unlikely to clash; a 21st step of acquiring structured data of an electronic medical record template relating to a specific patient from the structured data of the electronic medical record template; and a 22nd step of inserting the structured data into the document template output in the 20th step using the medical-designated terms included in the structured data of the electronic medical record template relating to the specific patient as a key, to generate document data.

[0236] (Appendix 27) A method executed by a computer having a processor and a memory, wherein the memory stores a machine learning model that receives input speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data by performing kana-to-kanji conversion on the speech recognition data; the processor executes the following steps: a first step of accepting information input from a user; a second step of presenting the information entered in the first step to the user; a third step of determining when the user has finished entering the information accepted in the first step; a fourth step of inputting the information accepted in the first step into the machine learning model until the end of input is determined in the third step, and also inputting environmental information and information into the machine learning model, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0237] (Appendix 28) A method executed by a computer having a processor and a memory, wherein the memory stores a machine learning model that receives input of speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and outputs kanji-converted speech recognition data by performing kana-kanji conversion on the speech recognition data; the processor executes the following steps: a first step of accepting input of information about the user's customers from a user; a second step of presenting the information entered in the first step to the user; a sixth step of determining whether the user has finished entering the information accepted in the first step based on at least one of the punctuation marks included in the information, line feed codes, the number of characters in the information, and a predetermined period of interruption in the user's input; a fourth step of inputting the information accepted in the first step into the machine learning model until the end of input is determined in the sixth step, and inputting environmental information and customer information into the machine learning model as prompts, and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0238] (Appendix 29) A method executed by a computer having a processor and a memory, wherein the memory stores a machine learning model that receives speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input, performs kana-to-kanji conversion on the speech recognition data, and outputs kanji-converted speech recognition data, and the processor performs a seventh step of accepting input of utterance data related to the user's customer from a user, an eighth step of performing speech recognition on the utterance data accepted in the seventh step to obtain speech recognition data, a ninth step of presenting the speech recognition data acquired in the eighth step to the user, and a ninth step of presenting the utterance data accepted in the seventh step to the user. a tenth step of determining whether the user's input of data has ended based on at least one of punctuation marks, line feed codes, the number of characters of information, and a predetermined period of interruption of the user's input, which are included in the voice recognition data; an eleventh step of inputting the voice recognition data acquired in the eighth step into a machine learning model until the end of input is determined in the tenth step, inputting environmental information and customer information into the machine learning model as prompts, and causing the machine learning model to output kanji-converted voice recognition data corresponding to the input information; and a twelfth step of presenting the kanji-converted voice recognition data output in the eleventh step to the user.

[0239] (Appendix 30) A method executed by a computer having a processor and a memory, wherein the memory stores a machine learning model that receives speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input, and outputs kanji-converted speech recognition data by performing kana-kanji conversion on the speech recognition data, and the processor includes a thirteenth step of accepting input of utterance data from a user, a fourteenth step of performing speech recognition on the received utterance data in the thirteenth step to obtain speech recognition data, a fifteenth step of inputting the speech recognition data obtained in the fourteenth step to the machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and a sixteenth step of presenting the kanji-converted speech recognition data, which is the output of the fifteenth step, to the user. How to perform.

[0240] (Appendix 31) a machine learning model that receives the medical designated words as input and outputs a document template that includes the medical designated words as character strings that are unlikely to clash; a 19th step of acquiring the structured data of the electronic medical record template from the memory; a 20th step of inputting the medical designated words included in the structured data of the electronic medical record template acquired in the 19th step into the machine learning model and outputting a document template that includes the medical designated words as character strings that are unlikely to clash; a 21st step of acquiring structured data of the electronic medical record template related to a specific patient from the structured data of the electronic medical record template; and a 22nd step of inserting the structured data into the document template output in the 20th step using the medical designated words included in the structured data of the electronic medical record template related to the specific patient as a key, thereby generating document data. How to perform. [Explanation of symbols]

[0241] 1...electronic medical record system, 10...terminal device, 20...server, 25...memory, 26...storage, 29...processor, 201...communication unit, 202...storage unit, 203...control unit, 2021...application program, 2023...electronic medical record data, 2024...electronic medical record template, 2025...document template, 2026...utterance data, 2027...speech recognition data, 2028...kanji-converted speech recognition data, 2029...teaching data, 2031...reception control module, 2032...transmission control module, 2033...speech recognition module, 2034...prompt generation module, 2035...generation model input / output module, 2036...presentation control unit, 2037...electronic medical record data generation module, 2040...machine learning model, 2041...medical designated word data, 2022...electronic medical record DB

Claims

1. A program for operating a computer having a processor and a memory, The program causes the processor to: A first step of accepting input of utterance data from a user; a second step of presenting to the user speech recognition data obtained by speech recognition of the speech data input in the first step, the speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a third step of determining whether the user has finished inputting the utterance data received in the first step; a fourth step of analyzing the speech data received in the first step until the end of input is determined in the third step, and acquiring converted speech recognition data in which either kana-kanji conversion or typo correction has been performed on the speech recognition data by analyzing at least one of (i) environmental information of the user, (ii) a document before the speech recognition data was acquired, and (iii) information about the user's customers, and (iv) the speech recognition data; and a fifth step of presenting the converted speech recognition data acquired in the fourth step to the user.

2. A program for operating a computer having a processor and a memory, The program causes the processor to: a first step of accepting input of information about the user's customers from the user; a second step of presenting to the user voice recognition data obtained by voice recognition of the information about the user's customer input in the first step, the voice recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a sixth step of determining whether the user has finished inputting the information about the user's customer that has been accepted in the first step based on at least one of punctuation marks, line feed codes, the number of characters in the information about the user's customer, and a predetermined period of interruption in the input by the user; a fourth step of analyzing information about the user's customer received in the first step until the end of input is determined in the sixth step, and analyzing (I) the user's environmental information, (II) at least one of the document before the voice recognition data was acquired, and (III) the voice recognition data, thereby acquiring converted voice recognition data in which the voice recognition data has been subjected to either kana-kanji conversion or correction of typos; and a fifth step of presenting the converted speech recognition data acquired in the fourth step to the user.

3. A program for operating a computer having a processor and a memory, The program causes the processor to: a seventh step of accepting input of utterance data relating to the user's customer from the user; an eighth step of performing speech recognition on the speech data received in the seventh step to obtain speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a ninth step of presenting the speech recognition data acquired in the eighth step to the user; a tenth step of determining whether the user has finished inputting the utterance data received in the seventh step based on at least one of punctuation marks included in the speech recognition data, a line feed code, the number of characters in the speech recognition data, and a predetermined period of interruption of the user's input; an eleventh step of analyzing the speech recognition data acquired in the eighth step until the end of input is determined in the tenth step, and analyzing at least one of the user's environmental information and the document before the speech recognition data was acquired, thereby acquiring converted speech recognition data in which the speech recognition data has been subjected to either kana-to-kanji conversion or correction of typos; and a twelfth step of presenting the converted speech recognition data acquired in the eleventh step to the user.

4. 4. The program according to claim 2, wherein the user is a medical professional and the customer is a patient whom the user is treating.

5. The user is a medical professional, and the user's environment information is information about the medical department to which the user belongs. The program according to any one of claims 1 to 3.

6. The program according to claim 4 , wherein the information about the user's customer is medical record information and / or a medical treatment summary from the customer's previous visit.

7. An information processing device including a processor and a memory, The processor: A first step of accepting input of utterance data from a user; a second step of presenting to the user speech recognition data obtained by speech recognition of the speech data input in the first step, the speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a third step of determining whether the user has finished inputting the utterance data received in the first step; a fourth step of analyzing the speech data received in the first step until the end of input is determined in the third step, and acquiring converted speech recognition data in which either kana-kanji conversion or typo correction has been performed on the speech recognition data by analyzing at least one of (i) environmental information of the user, (ii) a document before the speech recognition data was acquired, and (iii) information about the user's customers, and (iv) the speech recognition data; and a fifth step of presenting the converted speech recognition data acquired in the fourth step to the user.

8. An information processing device including a processor and a memory, The processor: a first step of accepting input of information about the user's customers from the user; a second step of presenting to the user voice recognition data obtained by voice recognition of the information about the user's customer input in the first step, the voice recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a sixth step of determining whether the user has finished inputting the information about the user's customer that has been accepted in the first step based on at least one of punctuation marks, line feed codes, the number of characters in the information about the user's customer, and a predetermined period of interruption in the input by the user; a fourth step of analyzing information about the user's customer received in the first step until the end of input is determined in the sixth step, and analyzing (I) the user's environmental information, (II) at least one of the document before the voice recognition data was acquired, and (III) the voice recognition data, thereby acquiring converted voice recognition data in which the voice recognition data has been subjected to either kana-kanji conversion or correction of typos; and a fifth step of presenting the converted speech recognition data acquired in the fourth step to the user.

9. An information processing device including a processor and a memory, The processor: a seventh step of accepting input of utterance data relating to the user's customer from the user; an eighth step of performing speech recognition on the speech data received in the seventh step to obtain speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a ninth step of presenting the speech recognition data acquired in the eighth step to the user; a tenth step of determining whether the user has finished inputting the utterance data received in the seventh step based on at least one of punctuation marks included in the speech recognition data, a line feed code, the number of characters in the speech recognition data, and a predetermined period of interruption of the user's input; an eleventh step of analyzing the speech recognition data acquired in the eighth step until the end of input is determined in the tenth step, and analyzing at least one of the user's environmental information and the document before the speech recognition data was acquired, thereby acquiring converted speech recognition data in which the speech recognition data has been subjected to either kana-to-kanji conversion or correction of typos; and a twelfth step of presenting the converted speech recognition data acquired in the eleventh step to the user.

10. 1. A method implemented by a computer having a processor and a memory, comprising: The processor: A first step of accepting input of utterance data from a user; a second step of presenting to the user speech recognition data obtained by speech recognition of the speech data input in the first step, the speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a third step of determining whether the user has finished inputting the utterance data received in the first step; a fourth step of analyzing the speech data received in the first step until the end of input is determined in the third step, and acquiring converted speech recognition data in which either kana-kanji conversion or typo correction has been performed on the speech recognition data by analyzing at least one of (i) environmental information of the user, (ii) a document before the speech recognition data was acquired, and (iii) information about the user's customers, and (iv) the speech recognition data; and a fifth step of presenting the converted speech recognition data obtained in the fourth step to the user.

11. 1. A method implemented by a computer having a processor and a memory, comprising: The processor: a first step of accepting input of information about the user's customers from the user; a second step of presenting to the user voice recognition data obtained by voice recognition of the information about the user's customer input in the first step, the voice recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a sixth step of determining whether the user has finished inputting the information about the user's customer that has been accepted in the first step based on at least one of punctuation marks, line feed codes, the number of characters in the information about the user's customer, and a predetermined period of interruption in the input by the user; a fourth step of analyzing information about the user's customer received in the first step until the end of input is determined in the sixth step, and analyzing (I) the user's environmental information, (II) at least one of the document before the voice recognition data was acquired, and (III) the voice recognition data, thereby acquiring converted voice recognition data in which the voice recognition data has been subjected to either kana-kanji conversion or correction of typos; and a fifth step of presenting the converted speech recognition data obtained in the fourth step to the user.

12. 1. A method implemented by a computer having a processor and a memory, comprising: The processor: a seventh step of accepting input of utterance data relating to the user's customer from the user; an eighth step of performing speech recognition on the speech data received in the seventh step to obtain speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks; a ninth step of presenting the speech recognition data acquired in the eighth step to the user; a tenth step of determining whether the user has finished inputting the utterance data received in the seventh step based on at least one of punctuation marks included in the speech recognition data, a line feed code, the number of characters in the speech recognition data, and a predetermined period of interruption of the user's input; an eleventh step of analyzing the speech recognition data acquired in the eighth step until the end of input is determined in the tenth step, and analyzing at least one of the user's environmental information and the document before the speech recognition data was acquired, thereby acquiring converted speech recognition data in which the speech recognition data has been subjected to either kana-to-kanji conversion or correction of typos; and a twelfth step of presenting the converted speech recognition data obtained in the eleventh step to the user.

Images