Program, information processing device, and method

Abstract

The aim is to streamline the process of recording information in electronic medical records based on voice data. Simultaneously, it will be delivered to medical institutions throughout Japan in a mass-customized format. [Solution] A program for operating a computer comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, and the program causes the processor to accept utterance data input from the user, to perform speech recognition on the accepted utterance data to obtain speech recognition data, to input the obtained speech recognition data into the machine learning model, to cause the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and to present the kanji-converted speech recognition data to the user.

Description

Technical Field

[0001] The present disclosure relates to a program, an information processing apparatus, and a method.

Background Art

[0002] An electronic medical record in which a doctor electronically records the content and results of an interview with a patient by voice or keyboard input and further electronically records the history of medical acts performed on the patient is known. The recorded content of the electronic medical record may be created according to an electronic medical record template.

[0003] As a technology related to the above-described technology, there is a technology disclosed in Patent Document 1.

[0004] Patent Document 1 discloses a technology related to a medical support apparatus. In the medical support apparatus, an input item display means displays input items on a display. An input item selection means selects one of the plurality of input items. A voice recognition means performs voice recognition of the input voice using the selected dictionary and extracts word candidates for the voice. A word candidate display means displays the extracted word candidates on the display. A selection operation reception means receives a selection operation of one word candidate from the word candidates. A storage control means stores the selected one word candidate in a storage means as an answer to the selected one input item.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The technology described in Patent Document 1 uses a specialized dictionary in the medical field to perform speech recognition processing. However, there are many homophones in medical template inputs, and even now there are certain limitations to the accuracy of speech recognition processing. As a result, doctors and other medical professionals may need to make corrections to the content entered into the electronic medical record template as a result of speech recognition processing.

[0007] Therefore, this disclosure has been made to solve the above-mentioned problems, and its purpose is to provide a technology that reduces the effort required to record information into an electronic medical record template based on voice data. [Means for solving the problem]

[0008] This is a program for operating a computer equipped with a processor and memory. The memory stores a machine learning model that takes speech recognition data containing at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji. The program causes the processor to execute the following steps: step 13, which receives utterance data input from the user; step 14, which performs speech recognition on the utterance data received in step 13 to obtain speech recognition data; step 15, which inputs the speech recognition data obtained in step 14 into the machine learning model and causes the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data; and step 16, which presents the kanji-converted speech recognition data, which is the output of step 15, to the user. [Effects of the Invention]

[0009] According to this disclosure, it is possible to reduce the effort required to record information into electronic medical record templates based on voice data. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows the overall configuration of a system according to one embodiment. [Figure 2]This figure shows the functional configuration of a terminal device according to one embodiment. [Figure 3] This figure shows the functional configuration of a server according to one embodiment. [Figure 4] This figure shows the data structure of an electronic medical record database according to one embodiment. [Figure 5] A flowchart showing an example of the processing flow in a system according to one embodiment. [Figure 6] This flowchart shows another example of the processing flow in a system according to one embodiment. [Figure 7] This is a schematic diagram showing an example of a screen displayed in a terminal device according to one embodiment. [Figure 8] This figure shows an example of the processing flow in a system according to one embodiment. [Figure 9] This figure shows another example of the processing flow in a system according to one embodiment. [Figure 10] This figure shows yet another example of the processing flow in a system according to one embodiment. [Figure 11] This is a schematic diagram illustrating another example of a screen displayed in a terminal device according to one embodiment. [Figure 12] This is a schematic diagram showing yet another example of a screen displayed in a terminal device according to one embodiment. [Figure 13] This is a schematic diagram showing yet another example of a screen displayed in a terminal device according to one embodiment. [Figure 14] This is a schematic diagram showing yet another example of a screen displayed in a terminal device according to one embodiment. [Figure 15] This is a schematic diagram showing yet another example of a screen displayed in a terminal device according to one embodiment. [Figure 16] This is a schematic diagram showing yet another example of a screen displayed in a terminal device according to one embodiment. [Figure 17] This figure illustrates the procedure for generating a letter of introduction using a system according to one embodiment. [Figure 18] This figure illustrates the procedure for generating a letter of introduction using a system according to one embodiment. [Figure 19] It is a flowchart showing an example of a processing flow in a system according to a modified example.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all the drawings for describing the embodiments, the same reference numerals are given to common components, and repeated descriptions are omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Also, not all of the components shown in the embodiments are necessarily essential components of the present disclosure. Also, each figure is a schematic diagram and is not necessarily drawn precisely.

[0012] Also, in the following description, the “processor” is one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may also be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be single-core or multi-core.

[0013] Also, at least one processor may be a processor in a broad sense such as a hardware circuit (for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0014] Also, in the following description, an expression such as “xxx table” may be used to describe information from which an output is obtained for an input, but this information may be data of any structure or a learning model such as a neural network that generates an output for an input. Therefore, “xxx table” can be referred to as “xxx information”.

[0015] Furthermore, in the following explanation, the structure of each table is just an example; one table may be divided into two or more tables, or all or part of two or more tables may be a single table.

[0016] Furthermore, in the following explanation, the subject of the process may sometimes be "program," but since a program is executed by a processor and performs defined processes using the memory and / or interface units as appropriate, the subject of the process may also be the processor (or a device such as a controller that has that processor).

[0017] The program may be installed on a device such as a computer, or it may reside on a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0018] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs, CPUs, conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0019] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0020] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0021] Furthermore, in the following explanation, identification numbers are used as identification information for various objects, but other types of identification information (for example, identifiers including letters or symbols) may also be used.

[0022] Furthermore, in the following explanations, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used, and when describing similar elements with distinction, the element's identification number (or reference code) may be used.

[0023] Furthermore, in the following explanation, only control lines and information lines deemed necessary for the explanation are shown, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0024] <0 System Overview> The system described herein is a system that records the contents of electronic medical records based on an electronic medical record template using speech recognition. In this specification, the contents of the electronic medical record are generated based on an electronic medical record template.

[0025] An electronic medical record (EMR) template is structured data that has input fields and associated input content. Here, structured data is data that is predefined and formatted to have a certain predetermined structure before being placed in storage. In contrast, unstructured data is data that is stored in plain text and is not processed until it is used. The input fields of an EMR are defined based on this EMR template. This system also includes an EMR template input assistance system that supports the input of template input fields by recreating the input fields of the EMR template in a different form such as a web form.

[0026] The input fields correspond to each field in the electronic medical record, and are relatively short, using medical terminology to help healthcare professionals identify which field they are referring to. The input content is what a doctor or healthcare professional enters into the input fields associated with this input content. The input content can be in the form of multiple-choice questions or open-ended questions, and the format varies. If the input is in the form of multiple-choice questions, the user selects one of the options (sometimes there is only one option), and if it is in the form of open-ended questions, a free-response text is entered. Note that in the case of input in the form of multiple-choice questions, the options include medical terminology. The electronic medical record template data is what a doctor or healthcare professional enters based on the electronic medical record template, and is the specific content of the input in the electronic medical record template.

[0027] In medical settings, doctors and healthcare professionals need to input large amounts of data into electronic medical records (EMRs) based on EMR templates. These EMR templates are structured data, some in multiple-choice format and others in open-ended format.

[0028] The input fields and content in electronic medical record templates are designed with the assumption that healthcare professionals will input, modify, and add to them, as well as view them. This means that the input fields and content must be based on medical knowledge and be medically accurate. However, the amount of information that healthcare professionals, including doctors, must record in electronic medical records is enormous, and the effort involved is considerable. For example, at the admission and discharge support center of a certain medical facility, the input content spans approximately six pages, and finding the appropriate field in the long, vertical profile section or assessment sheet of the electronic medical record and entering that information takes about 20 minutes per patient, which is a major reason for overtime work for nurses and medical office staff.

[0029] From this perspective, it is conceivable to use speech recognition to record the contents of electronic medical records. However, even at present, the accuracy of speech recognition processing is not sufficient. In particular, the contents to be recorded in electronic medical records include patient names, medically designated terms, etc., which are unlikely to be accurately converted into kana and kanji by general speech recognition engines.

[0030] Therefore, in the system disclosed herein, when recording the contents of an electronic medical record based on an electronic medical record template, a machine learning model is used to perform speech recognition processing. Preferably, medical information such as case information and medical notes are input to the machine learning model as prompts, and the contents of the electronic medical record are identified using the output from the machine learning model. By adopting such a configuration, the contents of the electronic medical record can be recorded efficiently and accurately using speech recognition technology.

[0031] Here, case information refers to information about a specific patient (not limited to a single person) that includes at least the current symptoms, and preferably includes the patient's age, sex, chief complaint as heard by the healthcare professional, current medical history and past medical history / family history, initial examination findings and laboratory test results for the patient, patient outcomes, imaging findings, and voice information. Case information may include information taken during face-to-face interviews by healthcare professionals, information entered by the patient on a questionnaire, and voice recordings.

[0032] To input case information into a system, it needs to be converted into electronic data (e.g., text data). For example, when a healthcare professional conducts an in-person interview, the information manually entered by the healthcare professional into the free-form field of the electronic medical record is included in the digitized case information. In addition, the information entered by the patient on the questionnaire prior to the interview by the healthcare professional also needs to be digitized. In recent years, there are systems that use tablet devices, etc., that allow for the electronic input of answers to questionnaire items, and the answers to questionnaires on these electronic questionnaires have already been digitized.

[0033] The system described in this disclosure uses a machine learning model that takes speech recognition data containing at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input and outputs kanji-converted speech recognition data obtained by performing kana-kanji conversion on the speech recognition data. This machine learning model may be a general-purpose large-scale language model such as ChatGPT, or it may be a machine learning model tuned for the system described in this disclosure.

[0034] When inputting medical information into a machine learning model, it is preferable to input the medical information as a prompt. Of course, it is also acceptable to input text other than medical information, supplemented with other text, into the machine learning model. Details regarding prompts will be described later.

[0035] It should be noted that, like human operators, the accuracy of voice recognition will not reach 100%. The fact that it is simpler than methods for verifying whether input has been entered correctly also significantly impacts user-friendliness.

[0036] Furthermore, the system relating to this disclosure generates document data based on electronic medical record data (including structured data of electronic medical record templates) whose recorded content has been finalized through the above-described process, preferably electronic medical record data for a specific patient, at the direction of a healthcare professional. The document data referred to herein is, for example, data for documents generated using a portion of the electronic medical record data, such as treatment summaries, referral letters, and reports to pharmaceutical companies.

[0037] The system described in this disclosure generates document templates for this document data using a machine learning model. The machine learning model takes medical terminology included in the input fields of the electronic medical record template as input and outputs a document template that includes this medical terminology as a string that is unlikely to cause collisions. This machine learning model may be a general-purpose large-scale language model such as ChatGPT, or it may be a machine learning model tuned for the system described in this disclosure.

[0038] <One Embodiment> <1 System Configuration Diagram> Figure 1 is a diagram showing the overall configuration of the electronic medical record system 1 of this embodiment. As shown in Figure 1, the electronic medical record system 1 includes a plurality of terminal devices (in Figure 1, terminal devices 10A and 10B are shown; hereinafter, they may be collectively referred to as "terminal device 10") and a server 20. The terminal devices 10 and the server 20 are connected to each other so as to be able to communicate with each other via a network 80. The network 80 is composed of a wired or wireless network. In this embodiment, the server 20 is a server that functions as a web server (including a cloud server) and exchanges information with the terminal devices 10 via web pages. In addition, the terminal devices 10 have a web page browser installed for viewing web pages, but a dedicated application for providing services of the server 20 may also be installed and configured to be viewable by the dedicated application.

[0039] The terminal device 10 is implemented using a stationary PC (Personal Computer), a laptop PC, or the like. Alternatively, the terminal device 10 may be a mobile device such as a tablet compatible with a mobile communication system or a smartphone.

[0040] Terminal device 10 is a device operated by a healthcare professional or the administrator of the electronic medical record system 1. Here, healthcare professionals include physicians, nurses, medical technologists with medical knowledge, etc. In the following explanation, unless otherwise specified, healthcare professionals will be considered to include the administrator of system 1.

[0041] Medical professionals use terminal device 10 to record electronic medical record content based on an electronic medical record template. In this process, medical professionals input spoken data into terminal device 10 and give instructions for input / correction / addition. Terminal device 10 sends the spoken data to server 20 and requests speech recognition processing and kana-kanji conversion processing (natural language processing) from server 20, and obtains kanji-converted speech recognition data from server 20. Next, medical professionals input / correct / add to the record content based on this kanji-converted speech recognition data using terminal device 10. Then, medical professionals give instructions to terminal device 10 to record the input content that has been input / corrected / added as the record content of the electronic medical record.

[0042] The terminal device 10 is connected to the server 20 via the network 80 in a communicative manner. The terminal device 10 connects to the network 80 by communicating with communication equipment such as a wireless base station 81 that supports communication standards such as 4G, 5G, and LTE (Long Term Evolution), and a wireless LAN router 82 that supports wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. As shown in Figure 1, the terminal device 10 includes a communication interface 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

[0043] The communication interface 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device (for example, a keyboard, touch panel, touchpad, mouse, or other pointing device) for receiving input operations from the user. The output device 14 is an output device (display, speaker, etc.) for presenting information to the user. The memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for saving data, such as flash memory or an HDD (Hard Disk Drive). The processor 19 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0044] Server 20 is managed by the administrator of the electronic medical record system 1 of this embodiment, and its stored contents can be modified / added / deleted as needed by medical professionals who are users of the terminal device 10. Server 20 is an electronic medical record device, and medical professionals in a medical facility can view the input items and contents of the electronic medical record via the terminal device 10 and modify / add to the input contents. It also accepts editing operations of the electronic medical record template performed by medical professionals via the terminal device 10, and modifies / adds / deletes the electronic medical record template based on these editing operations.

[0045] Server 20 is a computer connected to network 80. Server 20 includes a communication interface 22, an input / output interface 23, memory 25, storage 26, and a processor 29.

[0046] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices. Input / Output IF23 functions as an interface to an input device for receiving input operations from the user and an output device for presenting information to the user. Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). Storage 26 is a storage device for saving data, such as flash memory or HDD (Hard Disk Drive). Processor 29 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0047] <1.1 Functional configuration of terminal device 10> Figure 2 is a block diagram showing the functional configuration of the terminal device 10 that constitutes System 1 of this embodiment. As shown in Figure 2, the terminal device 10 includes a plurality of antennas (antenna 111, antenna 112), wireless communication units corresponding to each antenna (first wireless communication unit 121, second wireless communication unit 122), an input device 13 (including a keyboard 131 and a mouse 132), an audio processing unit 17 (including a microphone 171 and a speaker 172), a display 141 as an output device 14, a storage unit 180, and a control unit 190. The terminal device 10 also has functions and configurations not specifically shown in Figure 2 (for example, a battery for maintaining power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in Figure 2, each block included in the terminal device 10 is electrically connected by a bus or the like.

[0048] Antenna 111 radiates signals emitted by terminal device 10 as radio waves. Antenna 111 also receives radio waves from space and provides the received signals to first wireless communication unit 121.

[0049] Antenna 112 radiates signals emitted by terminal device 10 as radio waves. Antenna 112 also receives radio waves from space and provides the received signals to second wireless communication unit 122.

[0050] The first wireless communication unit 121 performs modulation and demodulation processing, etc., for the terminal device 10 to transmit and receive signals via the antenna 111 in order to communicate with other wireless devices. The second wireless communication unit 122 performs modulation and demodulation processing, etc., for the terminal device 10 to transmit and receive signals via the antenna 112 in order to communicate with other wireless devices. The first wireless communication unit 121 and the second wireless communication unit 122 are a communication module that includes a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation, demodulation, and frequency conversion of the wireless signals transmitted and received by the terminal device 10, and provide the received signal to the control unit 190.

[0051] The input device 13 has a mechanism for receiving user input operations. Specifically, the input device 13 includes a keyboard 131 and a mouse 132. The input device 13 may also be configured as a touchscreen that detects the user's contact position with the touch panel, for example, by using a capacitive touch panel.

[0052] The keyboard 131 accepts user input operations from the terminal device 10. The keyboard 131 is a device for character input and outputs the input character information as an input signal to the control unit 190.

[0053] The mouse 132 accepts user input operations from the terminal device 10. The mouse 132 is a pointing device for selecting objects displayed on the display 141, and outputs the selected position information on the screen and information indicating that a button is pressed as input signals to the control unit 190.

[0054] The audio processing unit 17 modulates and demodulates the audio signal. The audio processing unit 17 modulates the signal received from the microphone 171 and provides the modulated signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into sound and outputs the sound to the outside of the terminal device 10.

[0055] The display 141 displays data such as images, videos, and text in accordance with the control of the control unit 190. The display 141 is implemented by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0056] The storage unit 180 is composed of, for example, flash memory and stores data and programs used by the terminal device 10. In certain situations, the storage unit 180 stores user information 182.

[0057] User information 182 is information of a user who uses the terminal device 10 to perform the task of confirming the contents of the electronic medical record input by voice recognition processing, which is a function of System 1, and preferably is information of a medical professional who uses System 1 of this embodiment.

[0058] The control unit 190 controls the operation of the terminal device 10 by reading a program stored in the memory unit 180 and executing the instructions contained in the program. The control unit 190 is, for example, an application that is pre-installed on the terminal device 10. By operating according to the application program 181 stored in the memory unit 180, the control unit 190 performs the functions of an input operation receiving unit 191, a transmitting / receiving unit 192, a data processing unit 193, and a presentation control unit 194.

[0059] The input operation reception unit 191 processes input operations from the user to an input device such as a keyboard 131.

[0060] The transmitting / receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with an external device such as a server 20 in accordance with a communication protocol.

[0061] The data processing unit 193 performs calculations on the data received as input by the terminal device 10 according to the program and outputs the calculation results to memory or the like.

[0062] The presentation control unit 194 performs processing to present information to the user. The presentation control unit 194 performs processing such as displaying the display image on the display 141 and outputting sound to the speaker 172.

[0063] <1.2 Functional Configuration of Server 20> Figure 3 shows an example of the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0064] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0065] The memory unit 202 includes, for example, an electronic medical record database 2022, electronic medical record data 2023, an electronic medical record template 2024, a document template 2025, speech data 2026, speech recognition data 2027, kanji-converted speech recognition data 2028, training data 2029, a machine learning model 2040, medical term data 2041, and the like.

[0066] Electronic Medical Record DB2022 is a database for managing electronic medical record data for patients who have visited medical facilities using Server 20. Electronic Medical Record DB2022 may manage electronic medical record data from multiple medical facilities. Further details will be provided later.

[0067] Electronic medical record data 2023, when imported into the electronic medical record DB 2022, forms part of the recorded content of the electronic medical record. Electronic medical record data 2023 has input fields and input content associated with these input fields. There are no particular limitations on the data format of electronic medical record data 2023, but in this embodiment, electronic medical record data 2023 is data written in XAML (Extensible Application Markup Language) converted to JSON (JavaScript Object Notation) (JavaScript is a registered trademark). Preferably, the input fields of electronic medical record data 2023 are assigned identifiers such as sequences of numbers, and these identifiers also constitute electronic medical record data 2023. Electronic medical record data 2023 forms at least part of the medical information, including patient case information.

[0068] Electronic medical record template 2024 is a template used to generate electronic medical record data 2023. Electronic medical record template 2024 is structured data that defines input fields and the input content associated with these input fields. There are no particular limitations on the data format of electronic medical record template 2024, but in this embodiment, electronic medical record template 2024, like electronic medical record data 2023, is data described in XAML converted to JSON format. Similar to electronic medical record data 2023, it is preferable that the input fields of electronic medical record template 2024 are assigned identifiers such as sequences of numbers, and these identifiers also constitute electronic medical record template 2024.

[0069] Each electronic medical record template 2024 is associated with an identifier that identifies it. For example, the identifier for an electronic medical record template 2024 is a sequence of numbers of a predetermined number of digits.

[0070] Furthermore, in this embodiment, in the electronic medical record template 2024, multiple candidate answers (candidate input content) for medical questionnaire questions may be associated with each input item. In other words, the input content may be one candidate answer selected from multiple candidate answers.

[0071] Document template 2025 is a template for document data generated by the server 20 of this embodiment. In this embodiment, the document data may include, for example, a medical summary, a referral letter, and a report to a pharmaceutical company, and document template 2025 is provided for each type of document data.

[0072] The speech data 2026 is data recorded via the microphone 171 of the speech processing unit 17, which receives speech input from a medical professional to the terminal device 10. This data is then sent to the server 20 via the transmitting / receiving unit 192 of the control unit 190 of the terminal device 10. Preferably, the medical professional includes medical terms included in the medical term data 2041, which will be described later, in the speech data 2026. In other words, the medical professional inputs speech including medical terms into the microphone 171 of the terminal device 10.

[0073] The speech recognition data 2027 is speech recognition data obtained as a result of speech recognition performed by the speech recognition module 2033 of the control unit 203 (described later) based on the utterance data 2026. The speech recognition data 2027 in this embodiment is data that mixes hiragana, katakana, alphanumeric characters and punctuation marks, and preferably, space data or the like is inserted as a delimiter between these hiragana, katakana, alphanumeric characters and punctuation marks, making the hiragana and other characters easier to read.

[0074] The Kanji-converted speech recognition data 2028 is Kanji-converted speech recognition data output by the machine learning model 2040 as a result of the generative model input / output module 2035 (described later) inputting speech recognition data 2027 into the machine learning model 2040. The method for obtaining the Kanji-converted speech recognition data 2028 will be described in detail later.

[0075] The training data 2029 is the training data used to train the machine learning model 2040. The server 20 in this embodiment has two types of machine learning models 2040, and accordingly, it has training data 2029 for training each machine learning model 2040.

[0076] The training data 2029 includes speech recognition data 2027 containing at least one of hiragana, katakana, alphanumeric characters, and punctuation marks, and kanji-converted speech recognition data 2028 obtained by performing kana-kanji conversion on this speech recognition data. Preferably, the training data 2029 includes at least a portion of the electronic medical record data 2023.

[0077] The other training data 2029 includes medical term data 2041 and a document template 2025 that includes the medical terms contained in this medical term data 2041 as collision-free strings.

[0078] One form of machine learning model 2040 is a large-scale language model that, as exemplified by ChatGPT, takes natural language or vectors as input and outputs natural language or vectors in response to that natural language. In particular, in this embodiment, machine learning is performed using the respective training data 2029. One of the trained machine learning models 2040 is tuned to take speech recognition data 2027 containing at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input and output kanji-converted speech recognition data 2028, which is obtained by performing kana-kanji conversion on the input speech recognition data 2027. The other trained machine learning model 2040 is tuned to take medical terms included in medical term data 2041 as input and output a document template 2025 that includes these medical terms as collision-free strings.

[0079] Large-scale language models are generally also called generative AI, and are designed to automate the human process of writing text using machine learning. These models are first trained on a large amount of pre-training data, then aligned based on instructions and on-site feedback, and finally completed. These steps improve the quality of the text generated by the generative AI and allow for more accurate application of prompts (instructions given during use) to the generation process. Once a model is created, it becomes fixed. Instructions are then given to the model via prompts during use to control the generation process. The use of external search engines to obtain and input text for prompts is called RAG (Search-Augmented Generative).

[0080] The machine learning model 2040 according to this embodiment is, for example, a parameterized composite function composed of multiple functions. A parameterized composite function is defined by a combination of multiple tunable functions and parameters. The prediction model according to this embodiment may be any parameterized composite function that satisfies the above requirements, but is assumed to be a multi-layer network model (hereinafter referred to as a multi-layer network). A prediction model using a multi-layer network has an input layer, an output layer, and at least one intermediate or hidden layer between the input and output layers. The prediction model is intended to be used as a program module that is part of artificial intelligence software.

[0081] As the multilayer network according to this embodiment, for example, a deep neural network (DNN), which is a multilayer neural network targeted by deep learning, may be used. As the DNN, for example, a convolutional neural network (CNN) that targets images may be used.

[0082] Furthermore, the above is merely an example of a prediction model, and a prediction model may have other configurations. For example, the prediction model may be a rule-based model described by a function in which chief complaint information and environmental information are variables, and coefficients derived from past performance are attached to each variable.

[0083] The medical term data 2041 is used by the electronic medical record data generation module 2037 of the control unit 203 to identify input items related to the input content to be modified / added when modifying / adding to the input content of the electronic medical record based on the kanji-converted speech recognition data 2028. The medical terms referred to here are terms that are not limited to electronic medical records but are generally used by medical professionals when writing in medical records, and include at least so-called medical terms, as well as specialized terms used in input items. Preferably, the medical term data 2041 has a synonym dictionary for these medical terms, etc., and is used to identify and maintain the linking of items when modifying or updating. Medical terms are terms used in medical institutions to accurately describe medical terms, such as "past medical history," "present medical history," "medication history," "social history," "disease name," and "drug name." Also, synonyms for "present medical history" exist, such as "current symptoms" and "HPI," and when linking templates, synonyms may be used to guide or confirm the linking. The control unit 203 is realized when the processor 29 reads the application program 2021 stored in the memory unit 202 and executes the instructions contained in the application program 2021. By operating according to the application program 2021, the control unit 203 performs the functions shown as the reception control module 2031, transmission control module 2032, speech recognition module 2033, prompt generation module 2034, generation model input / output module 2035, presentation control unit 2036, and electronic medical record data generation module 2037.

[0084] The receive control module 2031 controls the process by which the server 20 receives signals from external devices according to a communication protocol.

[0085] The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

[0086] The speech recognition module 2033 acquires speech recognition data 2027 as an output result based on the speech data 2026 input by the medical professional via the microphone 171. As described in the description of speech recognition data 2027, the speech recognition module 2033 of this embodiment outputs speech recognition data 2027 which is a mixture of hiragana, katakana, alphanumeric characters and punctuation marks, and preferably, blank data is inserted between these hiragana, katakana, alphanumeric characters and punctuation marks as delimiters.

[0087] The prompt generation module 2034 generates prompts, which are text data to be input to the machine learning model 2040 in the memory unit 202, when various inputs are made to the machine learning model 2040 in the memory unit 202, and temporarily stores the generated prompts in the memory unit 202.

[0088] There are no particular restrictions on the prompts generated by the prompt generation module 2034, but as an example, the prompt generation module 2034 in this embodiment may generate medical information related to the content of voice input by a medical professional. For example, if the voice input is for inputting electronic medical record template data for a specific patient, then the prompt will be at least a part of the electronic medical record template data for this specific patient that has been accumulated in the past. Alternatively, the prompts generated by the prompt generation module 2034 may be text about the medical department to which the medical professional using the terminal device 10 belongs. The electronic medical record template data included in the prompt may include information for identifying the patient (ID), past electronic medical record data 2023, and a medical summary from when the patient was hospitalized. These can be collectively referred to as the medical history of a specific patient. Furthermore, the prompts generated by the prompt generation module 2034 may also include prompts that make the output from the machine learning model 2040 appropriate. Specific examples of such prompts will be described in detail later using screen examples, etc.

[0089] The generative model input / output module 2035 inputs the speech recognition data 2027 obtained as a result of speech recognition processing by the speech recognition module 2033, and the prompts generated by the prompt generation module 2034, into the machine learning model 2040. The machine learning model 2040 then outputs the kanji-converted speech recognition data 2028 corresponding to the speech recognition data 2027 input into the machine learning model 2040.

[0090] Alternatively, the generative model input / output module 2035 inputs medical terminology included in the medical terminology data 2041, as well as prompts generated by the prompt generation module 2034, into the machine learning model 2040. From this machine learning model 2040, a document template 2025 is output that includes the medical terminology input into the machine learning model 2040 as a string that is unlikely to cause collisions.

[0091] In this process, the generative model input / output module 2035 generates document data by inserting structured data into the document template 2025 output from the machine learning model 2040, using medical designation terms contained in the structured data of the electronic medical record template 2024 relating to a specific patient as keys. The generated document data is temporarily stored in the storage unit 202.

[0092] The presentation control unit 2036, in cooperation with the transmission control module 2032, transmits the output results of the speech recognition module 2033, the prompt generation module 2034, the generation model input / output module 2035, and the electronic medical record data generation module 2037 (described later) to a terminal device 10 operated by a medical professional, and displays them on the display 141 of the terminal device 10.

[0093] In this case, the presentation control unit 2036 changes the presentation method of the electronic medical record template 2024, which is otherwise identified by the electronic medical record data generation module 2037, and presents the recorded content to be recorded in the electronic medical record template 2024.

[0094] Alternatively, the presentation control unit 2036 presents the document template 2025, which is the output result of the machine learning model 2040, by changing the presentation method between the structured data inserted into the document template 2025 by the generation model input / output module 2035 and the document data other than the structured data inserted into the document template 2025.

[0095] Furthermore, the presentation control unit 2036 presents pairs of structured data input items and corresponding input content that the generation model input / output module 2035 has inserted into the document template 2025. At this time, the presentation control unit 2036 accepts selection inputs from the paired structured data input items and corresponding input content that the generation model input / output module 2035 should insert into the document data it generates. The generation model input / output module 2035 then inserts the selected input items and corresponding input content into the document data.

[0096] The electronic medical record data generation module 2037 identifies the record content to be recorded in the electronic medical record template 2024 based on the medical-specific terms included in the kanji-converted speech recognition data 2028. Then, the electronic medical record data generation module 2037 appends to and updates the electronic medical record template 2024 based on the identified record content.

[0097] <2 Data Structure> Figure 4 shows the data structure of the database stored by server 20. Note that Figure 4 is an example and does not exclude any data not shown.

[0098] The database shown in Figure 4 is a relational database, which manages data sets called tables, which are structurally defined by rows and columns, by relating them to each other. In a database, tables are called tables, the columns of a table are called columns, and the rows of a table are called records. In a relational database, relationships can be established and linked between tables.

[0099] Typically, each table has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit 203 of the server 20 can instruct the processor 29 to add, delete, or update records in specific tables stored in the storage unit 202, according to various programs.

[0100] Figure 4 shows the data structure of the electronic medical record DB2022. As shown in Figure 4, each record in the electronic medical record DB2022 includes, for example, the fields "Electronic Medical Record ID", "Patient ID", "Department ID", and "Electronic Medical Record Data". Each field in the electronic medical record DB2022 is entered by the electronic medical record template data recording module (not shown in the figure) when it generates the electronic medical record data 2023. The information stored in the electronic medical record DB2022 can be changed and updated as needed.

[0101] The item "Electronic Medical Record ID" is an ID used to identify the electronic medical record managed by System 1 (especially Server 20) of this embodiment. The item "Patient ID" is an ID used to identify the patient whose medical information is managed by the electronic medical record identified by the item "Electronic Medical Record ID". The item "Department ID" is an ID used to identify the department whose medical information is managed by the electronic medical record identified by the item "Electronic Medical Record ID". The item "Electronic Medical Record Data" is information regarding the file name of the electronic medical record data 2023 related to the electronic medical record identified by the item "Electronic Medical Record ID".

[0102] <3 Example of Operation> The following describes an example of the operation of the terminal device 10 and the server 20.

[0103] Figure 5 is a flowchart illustrating an example of the operation of terminal device 10. Figure 5 is a flowchart illustrating an example of the operation when an operator of terminal device 10 inputs / modifies / adds to the electronic medical record template 2024 using voice input.

[0104] In step S500, a medical professional performing input on the electronic medical record template 2024 preferably makes voice input including medically designated terms into the speaker 172 of the terminal device 10 that they operate. The data processing unit 193 of the terminal device 10 converts the input voice into speech data, which is digital data, and sends this speech data to the server 20 via the transmitting / receiving unit 192. Next, the control unit 203 of the server 20 receives the speech data sent from the terminal device 10 via the receiving control module 2031 and stores it in the storage unit 202. Specifically, for example, the control unit 203 receives the speech data sent from the terminal device 10 via the receiving control module 2031 and the speech recognition module 2033 and stores it in the storage unit 202 as speech data 2026.

[0105] Next, in step S501, the control unit 203 performs real-time speech recognition using the speech data 2026 received in step S500, and displays the speech recognition result in real time on the display 141 of the terminal device 10. Specifically, for example, the control unit 203 uses the speech recognition module 2033 to perform real-time speech recognition using the speech data 2026 received in step S500, and the presentation control unit 2036 displays the speech recognition result on the display 141 of the terminal device 10. As already mentioned, the speech recognition result in step S501 is data that mixes hiragana, katakana, alphanumeric characters and punctuation marks, and preferably, blank data is inserted as delimiters between these hiragana, katakana, alphanumeric characters and punctuation marks. Even if the quality of the real-time input conversion to kanji conversion etc. is poor, the user can see that input has been made and feel at ease. Note that the speech recognition results in step S501 may include kanji characters in data that contains a mixture of hiragana, katakana, alphanumeric characters, and punctuation. The important thing about this conversion is that it is a system that prioritizes time over quality, considering the two axes of time and quality.

[0106] Next, in step S502, the control unit 203 terminates the speech recognition process and confirms the speech recognition data 2027, which is the result of the speech recognition process, triggered by a speech recognition termination instruction input from a medical professional operating the terminal device 10, or by a certain period of interruption of voice input using the terminal device 10. Specifically, for example, the control unit 203 terminates the speech recognition process by the speech recognition module 2033 and confirms the speech recognition data 2027, which is the result of the speech recognition process, triggered by a speech recognition termination instruction input from a medical professional operating the terminal device 10, or by a certain period of interruption of voice input using the terminal device 10. The speech recognition module 2033 then stores the confirmed speech recognition data 2027 in the storage unit 202.

[0107] Next, in step S503, the control unit 203 generates prompts to be input to the machine learning model 2040 in step S504, described later, based on the patient's medical history related to the speech data 2026, the medical record information from the previous visit, the medical summary from the previous hospitalization, and the text of the referral letter. Specifically, for example, the control unit 203 uses the prompt generation module 2034 to generate prompts to be input to the machine learning model 2040 in step S504, described later, based on the patient's medical history related to the speech data 2026, the medical record information from the previous visit, the medical summary from the previous hospitalization, and the text of the referral letter. At this time, the prompt generation module 2034 searches the electronic medical record template 2024 using patient identification information (ID) and obtains the patient's medical history related to the speech data 2026, the medical record information from the previous visit, the medical summary from the previous hospitalization, and the text of the referral letter.

[0108] Then, in step S504, the control unit 203 inputs the prompt generated in step S503 and the speech recognition data 2027 confirmed in step S502 to the machine learning model 2040, and obtains the kanji-converted speech recognition data 2028 output from the machine learning model 2040, which is obtained by converting the speech recognition data 2027 from kana to kanji and / or correcting typos. Specifically, for example, the control unit 203 inputs the prompt generated in step S503 and the speech recognition data 2027 confirmed in step S502 to the machine learning model 2040 using the generation model input / output module 2035, and obtains the kanji-converted speech recognition data 2028 output from the machine learning model 2040, which is obtained by converting the speech recognition data 2027 from kana to kanji and / or correcting typos. The generation model input / output module 2035 then stores the obtained kanji-converted speech recognition data 2028 in the storage unit 202.

[0109] Generative AI generally has a somewhat slower output speed, but it is characterized by its ability to generate high-quality responses and text. Furthermore, the output results of generative AI can be controlled by adjusting the content of the prompts (instructions). Therefore, by providing prior instructions in the prompts regarding the context before and after the output text, the user's environment, and specific keywords (e.g., medical terms or proper nouns), generative AI can output highly accurate conversion or completion results that reflect the instructions.

[0110] The system described in this disclosure utilizes the characteristics of generative AI as described above. However, the transformation itself does not necessarily require the use of generative AI; for example, a general program may be used. Examples of general programs include general rule-based algorithms and hybrid programs that combine such algorithms with existing statistical language models. Alternatively, multiple types of general programs may be selected and combined according to the user's requirements (personality, attributes, etc.) and usage environment. Regardless of which of these is used, the system described in this disclosure prioritizes "quality" above all else in the transformation, out of the two axes of "time and quality."

[0111] Specifically, the system relating to this disclosure comprehensively incorporates the following information (a) to (f) and evaluates the quality of its generation or transformation.

[0112] (a) User environment information User environment information refers to information about the user's environment, such as the operating status of the devices the user is using, network communication status, and network access status. Specifically, for example, in a medical setting, this would include information about the usage environment of the electronic medical record system and information about the medical department to which the user belongs; in a restaurant, it would include information about the usage environment of POS registers and order management systems; and in various companies, including IT companies, it would include information about the usage environment of customer relationship management (CRM) and inventory management systems.

[0113] (i) Information regarding the user's intent The system identifies the ultimate purpose and completion requirements of the work or task that the user is attempting to perform (e.g., entering information into a medical record, adding comments to customer data, creating a report, etc.). In other words, the ultimate purpose and completion requirements identified from the work or task the user is attempting to perform serve as an example of information regarding the user's intentions. The system related to this disclosure then outputs information based on the results of this identification.

[0114] (c) Information about the user's customers For example, in a medical setting, patient information; in a restaurant, customer information; and in various types of businesses, information about business partners, all constitute information about the user's customers. The system disclosed here utilizes information about the user's customers, particularly names, ID information, preferences and allergy information, and purchase history, to perform accurate kanji and alphanumeric conversion. Furthermore, the system disclosed here utilizes a reference dictionary to prevent mistranslations of patient names and company names, especially in the automatic conversion of proper nouns.

[0115] (e) Information regarding user preferences Information regarding user preferences includes, for example, the user's writing style and tendency towards inconsistencies in notation, and whether or not abbreviations are acceptable to the user. The system relating to this disclosure uses such information regarding user preferences to prompt the user to make settings such as whether to correctly display medical terminology and abbreviations, or whether to leave company names and product names in their original language.

[0116] (e) Information regarding the user's habits If a user enters a phrase or notation rule specific to that user, or if the user has a habit of adding phonetic readings after a patient's name, such as "(Sato Taro)" when filling out a medical record, the system described in this disclosure will automatically complete the notation based on such habits.

[0117] (c) Information about the user's next process The system described in this disclosure takes input of how the generated or converted results will be used (such as transferring to templates, generating reports, outputting forms, or linking with other systems) and performs layout adjustments to match the template format or automatic conversion to a format that can be imported into existing systems.

[0118] By using multifaceted information such as (a) to (f) above as input, the system described in this disclosure can not only convert words and phrases but also generate high-quality results that are tailored to the diverse circumstances of the user. Specifically, the system described in this disclosure minimizes the effort required for the user to manually correct the output after it has been generated, such as converting the customer's name into accurate kanji characters or replacing alphabets with alphanumeric characters during speech recognition. Furthermore, the system described in this disclosure can also clearly specify designated terms in the medical field and keywords related to business flows through prompts, preventing the inclusion of unnecessary or inappropriate wording when instructing the generating AI.

[0119] The ultimate goal of the system described in this disclosure is to perform transformations that prioritize quality over time efficiency by utilizing information derived from multiple perspectives, such as "user environment," "user intent," "user's customers," "user preferences," "user habits," and "user's next process." This will increase the accuracy and validity of the generated or transformed text, and reduce the burden of corrections that users will have when actually using the system.

[0120] On the other hand, in step S550, the control unit 203 retrieves the electronic medical record template 2024 stored in the memory unit 202. Specifically, for example, the control unit 203 retrieves the electronic medical record template 2024 stored in the memory unit 202 using the generation model input / output module 2035.

[0121] Next, in step S551, the control unit 203 obtains text containing medical terminology associated with the electronic medical record template 2024 from the electronic medical record template 2024 called in step S550. Specifically, for example, the control unit 203 obtains text containing medical terminology associated with the electronic medical record template 2024 from the electronic medical record template 2024 called in step S550 using the prompt generation module 2034.

[0122] Next, in step S552, the control unit 203 obtains the patient's medical history related to the speech data 2026 obtained in step S503, the medical record information from the previous visit, the medical summary from the previous hospitalization, the text of the referral letter, and the Kanji-converted speech recognition data 2028 obtained in step S504. Specifically, for example, the control unit 203 obtains the patient's medical history related to the speech data 2026 obtained in step S503, the medical record information from the previous visit, the medical summary from the previous hospitalization, the text of the referral letter, and the Kanji-converted speech recognition data 2028 obtained in step S504, using the prompt generation module 2034.

[0123] Next, in step S553, the control unit 203 inputs the text obtained in step S551 or step S552 as a prompt to the machine learning model 2040 and obtains the document output from this machine learning model 2040. Specifically, for example, the control unit 203 inputs the text obtained in step S551 or step S552 as a prompt to the machine learning model 2040 using the prompt generation module 2034 and obtains the document output from this machine learning model 2040. The machine learning model 2040 used here is a machine learning model 2040 that takes the text obtained in step S551 or step S552 as input and outputs a document that can be written in the electronic medical record template 2024 based on the input text. Preferably, the output of this machine learning model 2040 includes medical terminology.

[0124] Next, in step S554, the control unit 203 links the document output from the machine learning model 2040 with the electronic medical record template 2024, using the medically designated terms contained in the document output from the machine learning model 2040 in step S553 as a key. Specifically, for example, the control unit 203 uses the electronic medical record data generation module 2037 to link the document output from the machine learning model 2040 with the electronic medical record template 2024, using the medically designated terms contained in the document output from the machine learning model 2040 in step S553 as a key. This operation corresponds to identifying the content to be recorded in the electronic medical record template 2024, which uses the medically designated terms as a key, based on the speech data 2026, speech recognition data 2027, and kanji-converted speech recognition data 2028.

[0125] Then, in step S555, the control unit 203 performs input / modification / overwriting operations on the electronic medical record template 2024 based on the record contents identified in step S554. Specifically, for example, the control unit 203 uses the electronic medical record data generation module 2037 to perform input / modification / overwriting operations on the electronic medical record template 2024 based on the record contents identified in step S554.

[0126] Furthermore, the processes in steps S500 to S504 of the flowchart in Figure 5 can be applied to situations other than when medical professionals input / modify / add to the electronic medical record template 2024 using voice input. Specifically, they can be applied, for example, when a restaurant owner or employee (user) operates the terminal device 10, or when a representative (user) from various companies, including IT companies, operates the terminal device 10.

[0127] Specifically, in step S500, the operator, such as the owner or employee of a restaurant, or a representative of a company, inputs voice into the speaker 172 of the terminal device 10 that they are operating. From this point onward, the various processes up to S502 are the same as in the case where the operator is a medical professional as described above.

[0128] Next, in step S503, the control unit 203 generates a prompt to be input to the machine learning model 2040 in step S504, based on the user's customer speech data 2026, and at least one of the user's environment information, information about the user's customers, and documents prior to the acquisition of speech recognition data. Here, "customer" includes, for example, patients in a medical setting, customers in a restaurant, and business partners in various companies. Information about the user's customers includes, for example, documents prior to the acquisition of speech recognition data 2027.

[0129] The customer information handled by each of the aforementioned industries / sectors includes, for example, the following:

[0130] (1) Medical setting (Customer: Patient) Examples include visit history, treatment history, contents of medical records from previous visits (documents prior to the acquisition of speech recognition data 2027), contents of medical summaries from previous hospitalizations (documents prior to the acquisition of speech recognition data 2027), and contents of referral letters. In particular, it is important to accurately convert the patient's name into kanji characters and to use alphanumeric characters for identification information (patient ID) to match with hospitalization history. Here, "previous" specifically refers to the visit / hospitalization before the acquisition of speech recognition data based on the patient's speech data 2026.

[0131] (2) Restaurants (customers: customers) Examples include visit history, order history, allergy or preference information, review comments, the contents of completed customer questionnaires (documents obtained before the acquisition of speech recognition data 2027), and the contents of referral letters from other stores. In particular, it is required that customer names (which may have the same pronunciation but different kanji spellings) be accurately and reliably converted using speech recognition. Specifically, completed customer questionnaires are those obtained before the acquisition of speech recognition data based on customer speech data 2026.

[0132] (3) Various companies (customers: business partners) This includes information related to sales negotiation history, product / service implementation history, support history, product inquiry logs, the contents of completed questionnaires by business partners (documents obtained before the acquisition of speech recognition data 2027), and text of referral letters from other companies. In particular, since company names and contact person names may be written in alphanumeric characters or katakana / Roman alphabet, it is necessary to perform character type conversion or spell correction at the speech recognition stage. Specifically, completed questionnaires by business partners are those obtained before the acquisition of speech recognition data based on the business partner's speech data 2026.

[0133] Specifically, for example, the prompt generation module 2034 generates a prompt to be input to the machine learning model 2040 in step S504 based on the user's customer utterance data 2026, as well as at least one of the customer's usage history, a summary of interactions with the customer, the contents of various reports from the previous usage, and the text of the introductory statement. In this case, the prompt generation module 2034 obtains various information about the customer, for example, using information to identify the customer (customer ID). Also, for example, the prompt generation module 2034 may take into account the user's environment information when generating the prompt. Furthermore, the utterance data 2026, the customer's usage history, a summary of interactions with the customer, the contents of various reports from the previous usage, the text of the introductory statement, and the user's environment information do not necessarily have to be input to the machine learning model 2040 in a manner that is incorporated into the prompt; for example, they may be input as explanatory variables of the machine learning model 2040.

[0134] The prompt generation module 2034 can, for example, in a medical setting, retrieve patient profiles and history, medical record information from previous visits, information about the treatment summary from previous visits, and the text of referral letters by searching the electronic medical record template 2024. Similarly, in a restaurant setting, the prompt generation module 2034 can retrieve order history, review history, and the text of referral letters from other establishments by searching an unillustrated customer database. Furthermore, in various types of businesses, the prompt generation module 2034 can retrieve the names of business partners, contact persons and their departments, negotiation history, inquiry logs, and the text of referral letters from other companies by searching an unillustrated customer information database.

[0135] Furthermore, accurate recognition and conversion of proper nouns of customers (patients, shop visitors, or business partners, etc.) are particularly important in the speech recognition process. For example, if a patient's name is uttered as "Sato," and candidate texts such as "Satou" or "sato" are generated, the prompt generation module 2034 will estimate the correct kanji conversion "Sato" and, if necessary, associate it with alphanumeric characters such as the patient ID "ID-12345." On the other hand, when a company name is uttered in Roman letters or katakana, the speech recognition system needs to accurately convert the name or organization name. To prevent misconversions and inconsistencies in spelling, it is recommended to utilize dictionary information and template information referenced by the machine learning model 2040.

[0136] In this way, by generating prompts based on speech data 2026 and information about the user's customers (including patient information), it becomes possible to optimize the dialogue and suggestion content output from the machine learning model 2040 (i.e., the Kanji-converted speech recognition data 2028) to match the customer's situation and attributes. Furthermore, by accurately transcribing and managing unique information such as customer names and customer IDs into text, it is possible to reduce the number of corrections required, improve the accuracy of history matching and the accuracy of acquiring additional information, and reduce matching and acquisition errors.

[0137] Next, in step S504, the control unit 203 may acquire the Kanji-converted speech recognition data 2028 without inputting the prompt generated in step S503 and the speech recognition data 2027 confirmed in step S502 into the machine learning model 2040.

[0138] Specifically, for example, the generative model input / output module 2035 may obtain the Kanji-converted speech recognition data 2028 by analyzing at least two combinations of the following using a rule-based algorithm: speech data 2026, speech recognition data 2027, customer usage history (information about the user's customers), summary of interactions with customers (information about the user's customers), contents of various reports from the previous usage (documents before the acquisition of speech recognition data 2027), introductory text, or user environment information. Alternatively, for example, the generative model input / output module 2035 may obtain the Kanji-converted speech recognition data 2028 by using both the machine learning model 2040 and a rule-based algorithm in combination.

[0139] Examples of rule-based algorithms used to acquire the Kanji-converted speech recognition data 2028 include the following well-known algorithms: (a) or (b).

[0140] (a) Algorithm for generating a sequence of phoneme data First, the audio signal is divided into short frames, and the audio features of each frame are calculated. These audio features include, for example, the frequency spectrum, formant frequency, and zero crossing rate. Next, the calculated audio features are analyzed, and candidate phonemes are estimated by applying thresholds or rules based on these features. Then, continuous data of the estimated phonemes is generated. In many cases, general signal processing techniques (e.g., filtering, Fourier transform) are used for the above-mentioned frame division of the audio signal and extraction of audio features.

[0141] (b) Algorithms for converting phoneme sequences into words or sentences First, a phoneme sequence is compared with a predefined word dictionary to extract matching word candidates. Next, grammatical rules or syntactic analysis are applied to score the candidate word sequences. Examples of grammatical rules and syntactic analysis include subject-verb-object sentence structure and phoneme connection rules. Then, based on the scoring results, the most likely word sequence or sentence is determined. Here, state transition models (e.g., finite state machines) or regular expressions are often used to apply grammatical rules. Furthermore, heuristic methods based on connection probability or confidence are often introduced in dictionary lookup and scoring.

[0142] Thus, speech recognition processing using rule-based algorithms is a process that combines speech features and grammatical rules, and is useful as a method for achieving speech recognition without using the machine learning model 2040.

[0143] To summarize, the control unit 203 can obtain the Kanji-converted speech recognition data 2028 by analyzing at least two combinations of (i) speech data 2026, (ii) user environment information, (iii) document before acquisition of speech recognition data, (iv) user customer information, or (v) speech recognition data 2027.

[0144] Figure 6 is a flowchart illustrating an example of the operation of server 20. Figure 6 is a flowchart illustrating an example of the operation when an operator of terminal device 10 generates document data using the electronic medical record template 2024 stored in server 20.

[0145] In step S600, the control unit 203 retrieves the electronic medical record template 2024 stored in the memory unit 202. Specifically, for example, the control unit 203 retrieves the electronic medical record template 2024 stored in the memory unit 202 using the generation model input / output module 2035.

[0146] Next, in step S601, the control unit 203 obtains data previously entered into the electronic medical record template 2024 from the electronic medical record template 2024 called in step S600. Specifically, for example, the control unit 203 obtains data previously entered into the electronic medical record template 2024 from the electronic medical record template 2024 called in step S600 using the generation model input / output module 2035.

[0147] On the other hand, in step S602, the control unit 203 generates a prompt containing medical designation terms associated with the electronic medical record template 2024, which was called in step S600. Specifically, for example, the control unit 203 generates a prompt containing medical designation terms associated with the electronic medical record template 2024, which was called in step S600, using the prompt generation module 2034.

[0148] Next, in step S603, the control unit 203 inputs the prompt generated in step S602 to the machine learning model 2040 and receives the output of a merge document template 2025 with form input functionality based on the input prompt from the machine learning model 2040. Specifically, for example, the control unit 203 inputs the prompt generated in step S602 to the machine learning model 2040 via the generation model input / output module 2035 and receives the output of a merge document template 2025 with form input functionality based on the input prompt from the machine learning model 2040. As described above, the document template 2025 output by the machine learning model 2040 in step S603 is a document template 2025 that includes the medical terminology contained in the prompt generated in step S602 as a string that is unlikely to cause collisions. Furthermore, the Mail Merge Template 2025 with form input functionality is a document template 2025 that has merge fields and merge forms (UI with form functionality that allows for easy selection using select boxes, checkboxes, free text fields, dropdowns, etc.) inserted into it.

[0149] In step S604, the control unit 203 selects the merge document template 2025 generated in step S603. Specifically, for example, the control unit 203 selects the merge document template 2025 generated in step S603 using the presentation control unit 2036.

[0150] Furthermore, when selecting a mail merge document, it is possible to display a special indicator, such as faint text, around the cursor to indicate that it is an input candidate, and to display mail merge document template candidates. If the template is not suitable, it will disappear when you start typing, or an instruction to change to a different template will be displayed. Depending on the content of the medical record immediately before, a template candidate may be selected, and the special UI indicator that it is an input candidate may be displayed as described above.

[0151] Next, in step S605, the control unit 203 creates document data based on the merge document template 2025 selected in step S604. Specifically, for example, the control unit 203 creates document data based on the merge document template 2025 selected in step S604 using the generation model input / output module 2035.

[0152] Next, in step S606, the control unit 203 highlights the inserted portion in the document data created in step S605. Specifically, for example, the control unit 203 displays the document data created in step S605 on the display 141 of the terminal device 10 using the presentation control unit 2036, and at this time, highlights the inserted portion in a way that makes it distinguishable from the rest of the data.

[0153] Next, in step S607, the control unit 203 prompts the user to input a form into the document data generated in step S605. Specifically, for example, the control unit 203 retrieves the content entered into the form by the medical professional via the terminal device 10 using the presentation control unit 2036, and displays the input content on the display 141 of the terminal device 10.

[0154] Next, in step S608, the control unit 203 confirms the contents entered into the form in step S607. Specifically, for example, the control unit 203 receives confirmation instructions for the form input contents displayed on the display 141 of the terminal device 10 from the medical professional operating the terminal device 10 via the presentation control unit 2036, and confirms the form input contents.

[0155] Then, in step S609, the control unit 203 inputs the document data confirmed in step S608 into the machine learning model 2040, and the machine learning model 2040 outputs document data with improved fluency. Specifically, for example, the control unit 203 inputs the document data confirmed in step S608 into the machine learning model 2040 using the generative model input / output module 2035, and the machine learning model 2040 outputs document data with improved fluency. The machine learning model 2040 here may be a general large-scale language model that has not undergone any special training.

[0156] <4 Screen Examples> The following describes an example of a screen output to the terminal device 10, with reference to Figures 7 to 18.

[0157] Figure 7 shows an example of a screen displayed on the terminal device 10's display 141, which shows the progress and results of the speech recognition processing in response to voice input made by the operator of the terminal device 10.

[0158] The screen 700 of the display 141 of the terminal device 10 is provided with an area 701 that displays the progress and results of speech recognition processing of voice input made by the operator of the terminal device 10. Within this area 701, the speech recognition processing results 702 by the speech recognition module 2033 and the machine learning model 2040 are displayed. In the example shown above in the figure, as already mentioned, the speech recognition module 2033 outputs speech recognition data 2027, which is the result of speech recognition processing, consisting of a mixture of hiragana, katakana, alphanumeric characters and punctuation marks, and preferably, blank data is inserted between these hiragana, katakana, alphanumeric characters and punctuation marks as delimiters. Therefore, the speech recognition processing result 702 shown above in the figure is data consisting of a mixture of hiragana, katakana, alphanumeric characters and punctuation marks, and furthermore, blank data is inserted between these hiragana, katakana, alphanumeric characters and punctuation marks as delimiters. The speech recognition processing results 702 are performed in real time and displayed on the display 141 in real time as well.

[0159] The prompt generation module 2034 obtains patient data related to voice input from the electronic medical record template 2024 and generates prompts using this data. For example, the prompt generation module 2034 obtains data from past medical record data, such as that shown in area 703 in the figure, and the generation model input / output module 2035 inputs this data as a prompt into the machine learning model 2040.

[0160] The speech recognition processing result 702 at the bottom of the figure is the Kanji-converted speech recognition data 2028, which is the output from the machine learning model 2040. This Kanji-converted speech recognition data 2028 demonstrates that accurate kana-to-Kanji conversion has been performed.

[0161] Figure 8 illustrates the procedure for generating a referral letter to another hospital, which is a document data, starting from the natural language text of a past referral letter stored in the Electronic Medical Record Template 2024.

[0162] First, the prompt generation module 2034 searches for the electronic medical record template 2024 and retrieves medical information suitable for creating document data from the medical information contained in the electronic medical record template 2024. In the illustrated example, this is a referral letter (natural language) from the previous medical setting.

[0163] Next, the prompt generation module 2034 designs prompts that will cause the machine learning model 2040 to generate a document template 2025 in which item names and values ​​such as medical terms are converted into a form separated by particles, symbols, etc. Then, the generation model input / output module 2035 inputs the prompt designed and output by the prompt generation module 2034 to the machine learning model 2040, causing the machine learning model 2040 to output a document template 2025 in which item names and values ​​such as medical terms are converted into a form separated by particles, symbols, etc.

[0164] Document template 2025 is a natural language document that recognizes medical terminology and symbols such as particles and structures it into table information. Alternatively, if the output from machine learning model 2040 is natural language, presentation control unit 2036 converts the natural language document into a table information document that recognizes medical terminology and symbols such as particles. When a healthcare professional operating the terminal device 10 generates a referral letter, the display 141 of the terminal device 10 displays a table-structured document template 2025 as shown in Figure 8. The healthcare professional then selects and inputs the items to be included in the referral letter.

[0165] Meanwhile, the prompt generation module 2034 generates a prompt that includes medical terminology and causes the machine learning model 2040 to output the desired document template 2025. This prompt includes medical terminology as collision-resistant symbols. The generation model input / output module 2035 inputs the prompt generated by the prompt generation module 2034 into the machine learning model 2040, causing the machine learning model 2040 to output the merged document template 2025, which includes medical terminology as collision-resistant symbols.

[0166] Subsequently, the generative model input / output module 2035 generates a referral letter, which is document data, by inserting the input content of the electronic medical record template 2024 using the medical terminology as a key.

[0167] Figure 9 shows examples of text representations corresponding to the merge template field function and the merge form function in Mail Merge Template 2025.

[0168] Figure 10 shows an example of a prompt generated by the prompt generation module 2034 when the speech recognition data 2027 output from the speech recognition module 2033 is input by the generation model input / output module 2035 to the machine learning model 2040 to obtain the kanji-converted speech recognition data 2028.

[0169] Figure 11 shows an example of a screen displayed on the display 141 of the terminal device 10. In the figure, the content to be reflected in the electronic medical record template 2024, i.e., the content to be entered (left side of the figure), and the voice recognition processing results (right side of the figure) are displayed. On the voice recognition processing results screen, a screen for displaying kanji conversion candidates based on voice input is superimposed.

[0170] Figure 12 shows an example of the screen displayed on the display 141 of the terminal device 10 when the operator of the terminal device 10 is performing input work on the electronic medical record template 2024.

[0171] On the left side of screen 1200 of the display 141 of the terminal device 10, screen 1201 is displayed, showing the input content identified as a result of speech recognition processing based on the input items of the electronic medical record template 2024 stored in the storage unit 202 of the server 20 and the utterance data 2026 from the operator. Also, on the upper right side of screen 1200 of the display 141, a voice input guidance screen 1202 is displayed, and on the lower right side of screen 1200, screen 1203 is displayed, showing the voice input results based on the utterance data 2026. Furthermore, at the bottom of screen 1200, buttons 1205 and 1206 are displayed to instruct the start of recording of the utterance data 2026 and the saving of the utterance data 2026.

[0172] The operator of the terminal device 10 gives an instruction to start recording or save the speech data 2026 by clicking buttons 1205 and 1206 via the input device 13.

[0173] Figure 13 shows a detailed view of screen 1201 shown in Figure 12. As already explained, screen 1300 (1201) is a screen that displays the input content identified as a result of speech recognition processing based on the input items of the electronic medical record template 2024 and the operator's speech data 2026.

[0174] The screen 1300 displayed on the display 141 of the terminal device 10 shows the input field 1301 of the electronic medical record template 2024 and the input content 1302 associated with this input field 1301. The input field 1301 also displays a number sequence 1303 to identify this input field 1301. As the speech recognition module 2033 processes the speech recognition, the voice input results are sequentially entered into the input content 1302. In addition, the input content 1302 of the electronic medical record template 2024 already contains some entered information, and voice input is used to add to / correct the already entered information. As shown in Figure 13, the already entered information is displayed in a specific color as "no correction," and the information added / corrected by voice input is displayed in a different color than the already entered information.

[0175] Figure 14 shows a detailed view of screen 1202 shown in Figure 12. As already explained, screen 1400 (1202) is a voice input guidance screen.

[0176] The screen 1400 of the display 141 of the terminal device 10 displays input field 1401, which is an input item of the electronic medical record template 2024 and includes medically designated terms, and an example 1402 of the input content that should be entered for input field 1401. In addition, if there are already entered items in the electronic medical record template 2024, the already entered items are displayed in the location of example 1402.

[0177] Figure 15 shows a screen that displays examples of kanji conversion candidates that pop up on screen 1200 when the operator of the terminal device 10 is performing voice input using screen 1200 as shown in Figure 12.

[0178] The screen 1500 of the display 141 of the terminal device 10 displays the hiragana notation 1501 and conversion candidate examples 1502 that were generated when the speech recognition module 2033 performed kanji conversion based on the speech data 2026. The operator of the terminal device 10 confirms the kanji conversion by selecting one of the conversion candidate examples 1502 using the input device 13.

[0179] Figure 16 shows an example of electronic medical record (EMR) template data provided by another medical institution for importing an EMR template. The structure of the EMR template data is the same as that shown in Figure 13, so a detailed explanation is omitted. Since it is EMR template data, it also includes profile information.

[0180] Figure 17 is a diagram illustrating the procedure for generating a referral letter document, which is generated by the server 20 based on electronic medical record template data.

[0181] The upper part of Figure 17 displays a list of electronic medical record template data (input content) for a specific patient. The operator of Server 20 selects from this list of input content which input content is necessary and which is unnecessary for generating the referral letter (necessary / unnecessary for the script).

[0182] Once the input content has been selected, Server 20 generates a script to be submitted to the text generation task of the large-scale language model (machine learning model 2040). The generated script is shown in the middle of Figure 17. At this point, as explained earlier, some numerical values ​​(in the example shown, numerical values ​​from the test results indicating blood sampling results) are replaced with dummy values ​​to verify that accurate text generation is being performed by the text generation task.

[0183] The results of feeding the script into a text generation task using a large-scale language model are shown in the lower panel of Figure 17.

[0184] Figure 18 shows an example of string replacement performed using the speech recognition module 2033 and the machine learning model 2040 in the procedure for generating the letter of introduction text shown in Figure 17.

[0185] <5 Effects of one embodiment> As described in detail above, according to System 1 of this embodiment, the contents of medical procedure records, such as electronic medical records, can be added to or modified using a simple procedure. This point will be explained in detail below.

[0186] The medical industry is one where mistakes are unacceptable. Therefore, there is a very high need for structured data using templates in the medical industry. Structured data can reduce medical errors. Electronic medical record templates allow for the standardization of workflows.

[0187] However, entering structured data into electronic medical records is a very time-consuming process. For example, at the admission and discharge support center of a certain medical facility, the input content typically spans six pages, and instructing staff on which fields to enter the data into and then actually entering it took about 20 minutes per patient. According to System 1 of this embodiment, this task could be reduced to about 5 minutes.

[0188] There are three reasons why using speech recognition to input structured data has not been used in the field until now. First, the accuracy of speech recognition is not sufficient. Second, there are easier input methods than speech recognition. Third, it is difficult to correct errors.

[0189] In System 1 according to this embodiment, the first problem, the accuracy of speech recognition, is resolved by limiting the usage scenarios, simultaneously displaying template input item names and candidate input content related to these input items, and narrowing down the patterns of input speech. In System 1 according to this disclosure, the WER (Word Error Rate) associated with speech recognition is reduced from 6% to approximately 2%.

[0190] The second problem was resolved by having the user input a simpler method—multiple-choice questions—before speech recognition, and then displaying the results. Speech recognition could then be used to correct or add to any insufficient information from the initial input.

[0191] The third problem, the difficulty in correcting mistakes, is resolved by displaying other suggestions after input, allowing users to select from those suggestions, and by making it easier to check what audio was used to generate the input.

[0192] In particular, in System 1 of this embodiment, since the input content in the multiple-choice format usually includes medically designated terms, speech recognition processing is performed using these medically designated terms as keys to identify at least the input content in the multiple-choice format, thereby further improving the accuracy of speech recognition.

[0193] <6 Variations> The embodiments described above are detailed explanations of the configuration in order to make this disclosure easier to understand, and are not necessarily limited to those comprising all the configurations described. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0194] As an example, in the system 1 of the embodiment described above, the machine learning model 2040 was located within the server 20, but it may also be located within an external server 40, as shown in Figure 1.

[0195] Furthermore, in the system 1 of the embodiment described above, accurate kana-to-kanji conversion of spoken data was performed using speech recognition processing and a machine learning model for tasks such as inputting, adding to, and overwriting the electronic medical record template 2024. However, the target of accurate kana-to-kanji conversion of spoken data using speech recognition processing and a machine learning model is not limited to the contents of the electronic medical record template 2024. Moreover, accurate kana-to-kanji conversion may be performed using a machine learning model on text data entered by the user using the input device 13 of the terminal device 10, etc., without going through speech recognition processing.

[0196] Furthermore, in the system 1 of the embodiment described above, the user operating the terminal device 10 manually instructed the timing of input and output of speech data. However, the server 20 or the like may autonomously make a determination based on at least one of the following: punctuation marks, line break codes, the number of characters in the text data entered by the user, and a predetermined time interruption in the user's input work.

[0197] Figure 19 shows a flowchart of the input content identification process for the Electronic Medical Record Template 2024, which starts with text input. When text is entered from the keyboard, "text indicating that the input is correct" is displayed in real time. Subsequently, the end of text input is determined based on some signal, such as the input of punctuation, line breaks, or no input for several seconds, and the "text entered from the keyboard" is input to the generation AI as a prompt, and typographical errors and kanji conversion are performed. Furthermore, accuracy is improved by adding electronic medical record information and surrounding information to the prompt at that time.

[0198] The operations shown in the flowchart of Figure 19 are substantially the same as those shown in the flowchart of Figure 5, so the explanation of those substantially identical parts will be omitted.

[0199] In step S1900, the user of the terminal device 10 inputs text data via the input device 13 by operating the keyboard 131. The operations in the subsequent steps S1901 to S1955 are the same as steps S501 to S555 in Figure 5.

[0200] In step S1903, the control unit 203 generates prompts to be input to the machine learning model 2040 based on the text data input from the input device 13. Specifically, for example, the control unit 203 uses the prompt generation module 2034 to correct typographical errors or perform appropriate kanji conversion, including medical terminology, on the aforementioned text data. As a result, the control unit 203 corrects the aforementioned text data with high accuracy and generates prompts to be input to the machine learning model 2040. Furthermore, the control unit 203 uses the prompt generation module 2034 to reflect patient information, disease name lists, dictionary data containing medical terms, medical term lists, etc., included in the electronic medical record template 2024 into the content of the prompts as needed. This results in even more accurate conversion results.

[0201] Then, in step S1904, the control unit 203 displays the Kanji-converted speech recognition data 2028 obtained from the machine learning model 2040 on a display device (not shown) provided by the server 20. This allows the user to not only confirm the conversion result (Kanji-converted speech recognition data 2028) through the display device, but also to manually edit the conversion result as needed. Meanwhile, the control unit 203 confirms the conversion result by receiving information that there are no problems with the conversion result, and reflects the confirmed conversion result in the electronic medical record template 2024.

[0202] As a result, even with simple text input from keyboard 131, the generating AI (machine learning model 2040) automatically corrects typos and performs kanji conversion based on triggers such as punctuation, line breaks, and interruptions in input. Furthermore, by reflecting patient information and disease lists already stored in the electronic medical record (information from the electronic medical record), as well as dictionary data containing medical terms and medical term lists (related information), in the prompts, more contextually accurate conversions can be achieved. This reduces input errors and inconsistencies in notation in medical settings, improving the reliability and efficiency of medical record creation.

[0203] Furthermore, the information input to the machine learning model 2040 may include the medical department to which the healthcare professional who owns the terminal device 10 belongs. The machine learning model 2040 may also be trained using information regarding the medical department.

[0204] Furthermore, the content of the prompt generated by the prompt generation module 2034 of the above-described embodiment may be input to the machine learning model 2040 by the generation model input / output module 2035, rather than as a prompt. If the machine learning model 2040 is, for example, a large-scale language model, an output close to the desired result can be obtained without using a prompt. However, if the desired result is not output, it is preferable to repeatedly provide the same input to the machine learning model 2040 and gradually obtain an output close to the desired result.

[0205] Furthermore, prior to speech recognition processing by the speech recognition module 2033, if a healthcare worker operating the terminal device 10 opens the electronic medical record template 2024 and views the patient's medical information related to the utterance data 2026, the prompt generation module 2034 monitors which patient's electronic medical record template 2024 is being viewed on each terminal device 10. Once speech recognition processing by the speech recognition module 2033 begins, the prompt generation module 2034 can use information to identify this patient to search for the electronic medical record template 2024 and obtain the patient's medical information in advance. This patient's medical information is then used for speech recognition processing by the machine learning model 2040.

[0206] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, in whole or in part, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0207] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0208] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0209] <7 Note> The details described in each of the above embodiments are noted below.

[0210] (Note 1) A program for operating a computer comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, and the program causes the processor to execute the following steps: a first step of receiving information input from a user; a second step of presenting the information input in the first step to the user; a third step of determining the end of the user's input of the information received in the first step; a fourth step of inputting the information received in the first step to the machine learning model, along with environmental information and other information, and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0211] (Note 2) A program for operating a computer comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the program causing the processor to execute: a first step of receiving input from a user regarding the user's customers; a second step of presenting the information entered in the first step to the user; a sixth step of determining the end of the user's input of the information received in the first step based on at least one of punctuation marks, newline characters, the number of characters in the information, and a predetermined time interruption in the user's input; a fourth step of inputting the information received in the first step to the machine learning model, while also inputting environmental information and customer information to the machine learning model as prompts, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0212] (Note 3) A program for operating a computer comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, and the program includes a seventh step in which the processor receives input of speech data about the user's customers from the user, an eighth step in which the speech data received in the seventh step is speech-recognized to obtain speech recognition data, a ninth step in which the speech recognition data obtained in the eighth step is presented to the user, and the program includes a seventh step in which the processor receives input of speech data about the user's customers from the user, an eighth step in which the speech data received in the seventh step is speech-recognized to obtain speech recognition data, a ninth step in which the program presents the speech recognition data obtained in the eighth step to the user, and the A program that performs the following steps: a 10th step in which the end of user input of spoken data is determined based on at least one of the punctuation marks, newline characters, number of characters in the information contained in the speech recognition data, and a predetermined time interruption in the user's input; an 11th step in which the speech recognition data acquired in step 8 is input into a machine learning model before the end of input is determined in step 10, and environmental information and customer information are input into the machine learning model as prompts, causing the machine learning model to output Kanji-converted speech recognition data corresponding to the input information; and a 12th step in which the Kanji-converted speech recognition data, which is the output of step 11, is presented to the user.

[0213] (Note 4) The program described in Appendix 2 or 3, where the user is a healthcare professional and the customer is a patient under the user's care.

[0214] (Note 5) The user is a healthcare professional, and the environmental information is related to the medical department to which the user belongs. The program is one of the programs described in Appendix 1-3.

[0215] (Note 6) The user's customer information is the medical record information and / or treatment summary from the customer's most recent visit, as described in Appendix 4 of the program.

[0216] (Note 7) A program for operating a computer comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters, and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, and the program causes the processor to execute a 13th step of receiving utterance data as input from a user, a 14th step of obtaining speech recognition data by speech recognition of the utterance data received in the 13th step, a 15th step of inputting the speech recognition data obtained in the 14th step into a machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and a 16th step of presenting the kanji-converted speech recognition data, which is the output of the 15th step, to the user.

[0217] (Note 8) The program, as described in Appendix 7, stores structured data of the electronic medical record template in memory, which is data that associates input items and input content of the electronic medical record template, with information for identifying the patient linked to the input content, and in step 15, searches the electronic medical record template using the information for identifying the patient, retrieves the input content from when the patient most recently visited or was hospitalized, inputs the speech recognition data into the machine learning model, and also inputs the input content from when the patient most recently visited or was hospitalized as a prompt into the machine learning model.

[0218] (Note 9) The program, as described in Appendix 7, stores structured data of an electronic medical record template in memory, the structured data is data that associates input items and input content of the electronic medical record template, and the input content is linked to information for identifying the patient, and the program causes the processor to execute a 24th step prior to the 13th step, which accepts the viewing of structured data of an electronic medical record template linked to the patient related to the speech data, and in the 13th step, searches the electronic medical record template using information for identifying the patient who has accepted the viewing of structured data of the electronic medical record template, and obtains the input content, which is the input content when the patient most recently visited or was hospitalized, and in the 15th step, inputs the speech recognition data into a machine learning model, and also inputs the input content when the patient most recently visited or was hospitalized into the machine learning model as a prompt.

[0219] (Note 10) The program, as described in Appendix 7, stores structured data of the electronic medical record template in memory, the structured data is data that associates input items and input content of the electronic medical record template, the input items include medical designation terms, the utterance data also includes medical designation terms, and the program causes the processor to perform a 17th step of further identifying the content to be recorded in the electronic medical record template based on the medical designation terms contained in the utterance data.

[0220] (Note 11) The speech data is for a specific patient, and in step 15, the program described in Appendix 10 inputs speech recognition data into a machine learning model, as well as an electronic medical record template for the specific patient as a prompt into the machine learning model.

[0221] (Note 12) The program described in Appendix 7 includes speech data relating to a specific patient, memory containing the patient's medical history, and in step 15, inputting speech recognition data into a machine learning model, as well as inputting the patient's medical history as a prompt into the machine learning model.

[0222] (Note 13) The program, as described in Appendix 10, further executes a 18th step in which the processor presents to the user an electronic medical record template containing the record contents identified in step 17, and further performs a change in the presentation manner of the identified record contents and the electronic medical record template excluding the record contents in step 18.

[0223] (Note 14) The program described in Appendix 10 specifies, in structured data of an electronic medical record template, the input content includes selection-type input content where the user selects an option, and in step 17, the program identifies the content to be recorded in the electronic medical record template based on at least the selection-type input content among the input content of the electronic medical record template.

[0224] (Note 15) In step 14, the speech recognition data is the program described in Appendix 7, in which blank data is inserted between each hiragana, katakana, alphanumeric character, and punctuation mark.

[0225] (Note 16) A program for operating a computer equipped with a processor and memory, wherein the memory stores structured data of an electronic medical record template, the structured data is data that associates input items and input content of the electronic medical record template, the input items include medically designated terms, and furthermore, the memory stores a machine learning model that takes the medically designated terms as input and outputs a document template that includes these medically designated terms as collision-resistant strings, and the program causes the processor to execute the following steps: step 19 of obtaining structured data of an electronic medical record template from memory, step 20 of inputting the medically designated terms contained in the structured data of the electronic medical record template obtained in step 19 into the machine learning model and causing it to output a document template that includes these medically designated terms as collision-resistant strings, step 21 of obtaining structured data of an electronic medical record template relating to a specific patient from among the structured data of the electronic medical record template.

[0226] (Note 17) The program then instructs the processor to perform a 23rd step in which it presents the document data generated in the 22nd step to the user. The program described in Appendix 16 that executes this program.

[0227] (Note 18) The program described in Appendix 17, which, in step 23, changes the presentation method of the structured data inserted into the document template and the document data other than the structured data inserted into the document template.

[0228] (Note 19) In step 23, the program described in Appendix 18 presents pairs of structured data input fields inserted into the document template and the corresponding input content.

[0229] (Note 20) In step 23, the program accepts a selection input of the input items to be inserted into the document data and the corresponding input content from the paired input items and corresponding input content of the structured data inserted into the document template, and the program causes the processor to execute step 24, which further inserts the selected input items and the corresponding input content into the document data, as described in Appendix 19.

[0230] (Note 21) The document templates include at least one of the following: a clinical summary, a referral letter, or a report to a pharmaceutical company, as described in Appendix 16.

[0231] (Note 22) Information processing device comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the processor performs the following steps: a first step of receiving information input from a user; a second step of presenting the information input in the first step to the user; a third step of determining the end of the user's input of the information received in the first step; a fourth step of inputting the information received in the first step to the machine learning model, along with environmental information and other information, and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0232] (Note 23) Information processing device comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the processor performs the following steps: a first step of receiving input from a user regarding the user's customers; a second step of presenting the information entered in the first step to the user; a sixth step of determining the end of the user's input of the information received in the first step based on at least one of punctuation marks, newline characters, the number of characters in the information, and a predetermined time interruption in the user's input; a fourth step of inputting the information received in the first step to the machine learning model, while also inputting environmental information and customer information as prompts to the machine learning model, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0233] (Note 24) An information processing device comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the processor performs a seventh step of receiving input of speech data about the user's customers from the user, an eighth step of obtaining speech recognition data by speech recognition of the speech data received in the seventh step, a ninth step of presenting the speech recognition data obtained in the eighth step to the user, and the user of the speech data received in the seventh step An information processing device that performs the following steps: a 10th step in which the end of input is determined based on at least one of the punctuation marks, newline characters, number of characters in the information contained in the speech recognition data, and a predetermined time interruption of user input; an 11th step in which the speech recognition data acquired in step 8 is input to a machine learning model before the end of input is determined in step 10, and environmental information and customer information are input to the machine learning model as prompts, causing the machine learning model to output Kanji-converted speech recognition data corresponding to the input information; and a 12th step in which the Kanji-converted speech recognition data, which is the output of step 11, is presented to the user.

[0234] (Note 25) An information processing device comprising a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, and the processor performs the following steps: a 13th step of receiving utterance data as input from a user; a 14th step of obtaining speech recognition data by speech recognition of the utterance data received in the 13th step; a 15th step of inputting the speech recognition data obtained in the 14th step into the machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data; and a 16th step of presenting the kanji-converted speech recognition data, which is the output of the 15th step, to the user.

[0235] (Note 26) Information processing device comprising a processor and memory, wherein the memory stores structured data of an electronic medical record template, the structured data is data relating input items and input content of the electronic medical record template, the input items include medically designated terms, and furthermore, the memory stores a machine learning model that takes the medically designated terms as input and outputs a document template that includes these medically designated terms as collision-resistant strings, the information processing device executing the following steps: step 19 of acquiring structured data of an electronic medical record template from memory; step 20 of inputting the medically designated terms contained in the structured data of the electronic medical record template acquired in step 19 into the machine learning model and causing it to output a document template that includes these medically designated terms as collision-resistant strings; step 21 of acquiring structured data of an electronic medical record template relating to a specific patient from among the structured data of the electronic medical record template.

[0236] (Note 27) A method performed by a computer having a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, and the processor performs the following steps: a first step of receiving information input from a user; a second step of presenting the information input in the first step to the user; a third step of determining the end of the user's input of the information received in the first step; a fourth step of inputting the information received in the first step to the machine learning model, along with environmental information and other information, and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information; and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0237] (Note 28) A method performed by a computer having a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the processor performs a first step of receiving input from a user regarding the user's customers, a second step of presenting the information entered in the first step to the user, a sixth step of determining when the user has finished entering the information received in the first step based on at least one of punctuation marks, newline characters, the number of characters in the information, and a predetermined time interruption in the user's input, a fourth step of inputting the information received in the first step to the machine learning model, and prompting the machine learning model to input environmental information and customer information as prompts, causing the machine learning model to output kanji-converted speech recognition data corresponding to the input information, and a fifth step of presenting the kanji-converted speech recognition data, which is the output of the fourth step, to the user.

[0238] (Note 29) A method performed by a computer having a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the processor takes the following steps: step 7, receiving input of speech data about the user's customers from the user; step 8, performing speech recognition on the speech data received in step 7 to obtain speech recognition data; step 9, presenting the speech recognition data obtained in step 8 to the user; and step 7, receiving the speech data A method comprising: a 10th step in which the end of user input is determined based on at least one of punctuation marks, newline characters, the number of characters in the information contained in the speech recognition data, and a predetermined time interruption in the user's input; an 11th step in which the speech recognition data acquired in step 8 is input to a machine learning model before the end of input is determined in step 10, and environmental information and customer information are input to the machine learning model as prompts, causing the machine learning model to output Kanji-converted speech recognition data corresponding to the input information; and a 12th step in which the Kanji-converted speech recognition data, which is the output of step 11, is presented to the user.

[0239] (Note 30) A method performed by a computer having a processor and memory, wherein the memory stores a machine learning model that takes speech recognition data including at least one of hiragana, katakana, alphanumeric characters and punctuation marks as input and outputs kanji-converted speech recognition data performed by converting the speech recognition data from kana to kanji, the processor performs a 13th step of receiving utterance data as input from the user, a 14th step of obtaining speech recognition data by speech recognition of the utterance data received in the 13th step, a 15th step of inputting the speech recognition data obtained in the 14th step into the machine learning model and causing the machine learning model to output kanji-converted speech recognition data corresponding to the input speech recognition data, and a 16th step of presenting the kanji-converted speech recognition data, which is the output of the 15th step, to the user. How to do it.

[0240] (Note 31) A method executed by a computer having a processor and memory, wherein the memory stores structured data of an electronic medical record template, the structured data is data that associates input items and input content of the electronic medical record template, the input items include medical designation terms, and furthermore, the memory stores a machine learning model that takes the medical designation terms as input and outputs a document template that includes these medical designation terms as collision-free strings, the processor performing the following steps: step 19: obtain the structured data of the electronic medical record template from memory; step 20: input the medical designation terms included in the structured data of the electronic medical record template obtained in step 19 into the machine learning model and output a document template that includes these medical designation terms as collision-free strings; step 21: obtain the structured data of the electronic medical record template relating to a specific patient from among the structured data of the electronic medical record template, and step 22: generate document data by inserting the structured data, using the medical designation terms included in the structured data of the electronic medical record template relating to the specific patient as keys, into the document template output in step 20. How to do it. [Explanation of symbols]

[0241] 1…Electronic medical record system, 10…Terminal device, 20…Server, 25…Memory, 26…Storage, 29…Processor, 201…Communication unit, 202…Storage unit, 203…Control unit, 2021…Application program, 2023…Electronic medical record data, 2024…Electronic medical record template, 2025…Document template, 2026…Speech data, 2027…Speech recognition data, 2028…Speech recognition data with kanji conversion, 2029…Training data, 2031…Receive control module, 2032…Transmit control module, 2033…Speech recognition module, 2034…Prompt generation module, 2035…Generative model input / output module, 2036…Presentation control unit, 2037…Electronic medical record data generation module, 2040…Machine learning model, 2041…Medical term data, 2022…Electronic medical record DB

Claims

1. A program for operating a computer that includes a processor and memory, The program is provided to the processor: The first step is to receive speech data input from the user, A second step involves presenting to the user speech recognition data obtained by speech recognition of the speech data input in the first step, which includes at least one of hiragana, katakana, alphanumeric characters, and punctuation marks. A third step is to determine the end of the user's input of the speech data received in the first step, The fourth step involves analyzing the speech data received in the first step before the end of input is determined in the third step, and obtaining converted speech recognition data by analyzing (i) the user's environment information, (ii) the document before the acquisition of the speech recognition data, and (iii) at least one of the user's customer information, and (iv) the speech recognition data, thereby performing either kana-to-kanji conversion or correction of misspellings on the speech recognition data. A program that performs a fifth step of presenting the converted speech recognition data obtained in the fourth step to the user.

2. A program for operating a computer that includes a processor and memory, The program is provided to the processor: The first step is to receive input from the user regarding the user's customers, A second step involves presenting to the user speech recognition data obtained by speech recognition of the customer information of the user that was entered in the first step, which includes at least one of hiragana, katakana, alphanumeric characters and punctuation marks. A sixth step in which the user's input of customer information received in the first step is determined based on at least one of the punctuation marks, newline characters, number of characters in the customer information, and a predetermined time interruption in the user's input, The fourth step involves analyzing the customer information of the user received in the first step before the end of input is determined by the sixth step, and obtaining converted speech recognition data by analyzing (I) the user's environmental information and (II) at least one of the document before the acquisition of the speech recognition data, and (III) the speech recognition data, thereby performing either kana-to-kanji conversion or correction of typographical errors on the speech recognition data. A program that performs a fifth step of presenting the converted speech recognition data obtained in the fourth step to the user.

3. A program for operating a computer that includes a processor and memory, The program is provided to the processor: A seventh step involves receiving input from the user regarding the user's customers, Step 8 involves performing speech recognition on the speech data received in step 7 to obtain speech recognition data that includes at least one of hiragana, katakana, alphanumeric characters, and punctuation marks. A ninth step involves presenting the speech recognition data acquired in the eighth step to the user, Step 10 determines the end of user input of the speech data received in step 7 based on at least one of punctuation marks, newline characters, the number of characters in the speech recognition data, and a predetermined time interruption of user input. Step 11 involves analyzing the speech recognition data acquired in step 8 before the end of input is determined by step 10, and analyzing at least one of the user's environment information and the document before the acquisition of the speech recognition data to obtain converted speech recognition data by performing either kana-kanji conversion or correction of typographical errors on the speech recognition data. A program that performs a 12th step of presenting the converted speech recognition data obtained in the 11th step to the user.

4. The program according to claim 2 or 3, wherein the user is a healthcare professional and the customer is a patient under the care of the user.

5. The user is a healthcare professional, and the user's environmental information is related to the medical department to which the user belongs. The program according to any one of claims 1 to 3.

6. The program according to claim 4, wherein the information relating to the user's customer is the medical record information and / or treatment summary from the customer's previous visit.

7. An information processing device comprising a processor and memory, The aforementioned processor, The first step is to receive speech data input from the user, A second step involves presenting to the user speech recognition data obtained by speech recognition of the speech data input in the first step, which includes at least one of hiragana, katakana, alphanumeric characters, and punctuation marks. A third step is to determine the end of the user's input of the speech data received in the first step, The fourth step involves analyzing the speech data received in the first step before the end of input is determined in the third step, and obtaining converted speech recognition data by analyzing (i) the user's environment information, (ii) the document before the acquisition of the speech recognition data, and (iii) at least one of the user's customer information, and (iv) the speech recognition data, thereby performing either kana-to-kanji conversion or correction of misspellings on the speech recognition data. An information processing device that performs a fifth step of presenting the converted speech recognition data obtained in the fourth step to the user.

8. An information processing device comprising a processor and memory, The aforementioned processor, The first step is to receive input from the user regarding the user's customers, A second step involves presenting to the user speech recognition data obtained by speech recognition of the customer information of the user that was entered in the first step, which includes at least one of hiragana, katakana, alphanumeric characters and punctuation marks. A sixth step in which the user's input of customer information received in the first step is determined based on at least one of the punctuation marks, newline characters, number of characters in the customer information, and a predetermined time interruption in the user's input, The fourth step involves analyzing the customer information of the user received in the first step before the end of input is determined by the sixth step, and obtaining converted speech recognition data by analyzing (I) the user's environmental information and (II) at least one of the document before the acquisition of the speech recognition data, and (III) the speech recognition data, thereby performing either kana-to-kanji conversion or correction of typographical errors on the speech recognition data. An information processing device that performs a fifth step of presenting the converted speech recognition data obtained in the fourth step to the user.

9. An information processing device comprising a processor and memory, The aforementioned processor, A seventh step involves receiving input from the user regarding the user's customers, Step 8 involves performing speech recognition on the speech data received in step 7 to obtain speech recognition data that includes at least one of hiragana, katakana, alphanumeric characters, and punctuation marks. A ninth step involves presenting the speech recognition data acquired in the eighth step to the user, Step 10 determines the end of user input of the speech data received in step 7 based on at least one of punctuation marks, newline characters, the number of characters in the speech recognition data, and a predetermined time interruption of user input. Step 11 involves analyzing the speech recognition data acquired in step 8 before the end of input is determined by step 10, and analyzing at least one of the user's environment information and the document before the acquisition of the speech recognition data to obtain converted speech recognition data by performing either kana-kanji conversion or correction of typographical errors on the speech recognition data. An information processing device that performs a 12th step of presenting the converted speech recognition data obtained in the 11th step to the user.

10. A method performed by a computer having a processor and memory, wherein The aforementioned processor, The first step is to receive speech data input from the user, A second step involves presenting to the user speech recognition data obtained by speech recognition of the speech data input in the first step, which includes at least one of hiragana, katakana, alphanumeric characters, and punctuation marks. A third step is to determine the end of the user's input of the speech data received in the first step, The fourth step involves analyzing the speech data received in the first step before the end of input is determined in the third step, and obtaining converted speech recognition data by analyzing (i) the user's environment information, (ii) the document before the acquisition of the speech recognition data, and (iii) at least one of the user's customer information, and (iv) the speech recognition data, thereby performing either kana-to-kanji conversion or correction of misspellings on the speech recognition data. A method comprising the steps of: a fourth step, the fourth step, the fourth step, the fifth step, the

11. A method performed by a computer having a processor and memory, wherein The aforementioned processor, The first step is to receive input from the user regarding the user's customers, A second step involves presenting to the user speech recognition data obtained by speech recognition of the customer information of the user that was entered in the first step, which includes at least one of hiragana, katakana, alphanumeric characters and punctuation marks. A sixth step in which the user's input of customer information received in the first step is determined based on at least one of the punctuation marks, newline characters, number of characters in the customer information, and a predetermined time interruption in the user's input, The fourth step involves analyzing the customer information of the user received in the first step before the end of input is determined by the sixth step, and obtaining converted speech recognition data by analyzing (I) the user's environmental information and (II) at least one of the document before the acquisition of the speech recognition data, and (III) the speech recognition data, thereby performing either kana-to-kanji conversion or correction of typographical errors on the speech recognition data. A method comprising the steps of: a fourth step, the fourth step, the fourth step, the fifth step, the

12. A method performed by a computer having a processor and memory, wherein The aforementioned processor, A seventh step involves receiving input from the user regarding the user's customers, Step 8 involves performing speech recognition on the speech data received in step 7 to obtain speech recognition data that includes at least one of hiragana, katakana, alphanumeric characters, and punctuation marks. A ninth step involves presenting the speech recognition data acquired in the eighth step to the user, Step 10 determines the end of user input of the speech data received in step 7 based on at least one of punctuation marks, newline characters, the number of characters in the speech recognition data, and a predetermined time interruption of user input. Step 11 involves analyzing the speech recognition data acquired in step 8 before the end of input is determined by step 10, and analyzing at least one of the user's environment information and the document before the acquisition of the speech recognition data to obtain converted speech recognition data by performing either kana-kanji conversion or correction of typographical errors on the speech recognition data. A method comprising the steps of: presenting the converted speech recognition data obtained in the 11th step to the user in the 12th step.

Images