Medical information processing device, medical information processing method, and program

Abstract

To provide a medical information processing device, a medical information processing method, and a program for assisting in the work of inputting medical information to a medical information processing system.SOLUTION: The medical information processing device acquires image data including a text indicating medical information, converts the text extracted from the image data and included in a text region into text data, displays the text region on top of the image data, receives selectin of a first text region from a plurality of text regions, combines the selected first region with a second text region into one text region, and selectably displays a plurality of candidates for medical information indicated by a text in the combined text region on the basis of the text data of the text included in the combined text region.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a medical information processing device, a medical information processing method, and a program, and more particularly to a technology for recognizing text containing medical information. [Background technology]

[0002] Medicine notebooks, prescriptions, etc. contain multiple pieces of medical information such as the name of the medication to be taken, dosage and administration, and when to take it. In medical settings, there are many situations where this medical information must be manually entered into medical information processing systems such as electronic medication history and electronic medical records, which is a significant hassle.

[0003] Even when medical information is stored on electronic media, there are situations where the lack of data linkage functionality means that the medical information must be manually entered into the medical information processing system, which is also a significant hassle. For example, before hospitalization, a screen shot of the patient's electronic medicine record app on their smartphone must be taken, and the names of the medications being taken, etc., must be manually entered into the medical information system based on the captured image.

[0004] In response to this, Patent Documents 1 and 2 describe a technique for recognizing characters recorded on a prescription by OCR (Optical Character Reader) processing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-2918 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-122360 Summary of the Invention [Problem to be solved by the invention]

[0006] Normally, the information entered into a medical information processing system must exactly match the master records registered in the drug name master, dosage and administration master, etc.

[0007] On the other hand, when character recognition is performed using OCR processing, there is a possibility that due to recognition fluctuations the recognition results may not perfectly match records such as the drug name master, dosage master, etc. In this case, a method is usually used in which a record with a high degree of match is obtained as a candidate by performing a brute force search between the text data (recognized character string) and the master record.

[0008] However, in cases where the OCR process results in a single piece of medical information being split into multiple character strings and recognized, or where a single piece of medical information is displayed across multiple lines on an electronic medication record app due to a small screen on a smartphone app, and therefore recognized as multiple text fields, the degree of match with records such as the drug name master and dosage master is very low, and simple matching-based methods cannot be used to address these cases.

[0009] Furthermore, medicine notebooks, prescriptions, etc. contain many character strings that are not to be input. Items that can be input include, for example, drug names, dosage and administration instructions, and when to take the medicine, while items that are not to be input include, for example, document titles, annotations, and item names. For this reason, users had to separate the set of character strings recognized by OCR processing into those that were to be input and those that were not. Furthermore, because it was unknown which medical item each element in the set of character strings recognized by OCR processing corresponded to, it was necessary to separate and specify which medical item each element corresponded to.

[0010] The present invention has been made in consideration of these circumstances, and aims to provide a medical information processing device, a medical information processing method, and a program that solve at least one of the above problems and assist in the task of inputting medical information into a medical information processing system. [Means for solving the problem]

[0011] In order to achieve the above object, a medical information processing device according to a first aspect of the present disclosure includes at least one processor and at least one memory that stores instructions to be executed by the processor, wherein the processor acquires image data including text indicating medical information, extracts a text area from the image data, converts the text included in the text area into text data, superimposes the text area on the image data, accepts selection of a first text area from the multiple superimposed text areas, integrates the selected first text area and a second text area consisting of one or more text areas different from the first text area into a single text area, and displays multiple selectable candidates of medical information indicated by the text in the integrated text area based on the text data of the text included in the integrated text area.

[0012] According to this aspect, it is possible to support the task of correcting recognition errors in OCR processing and inputting medical information into a medical information processing system.

[0013] A medical information processing device according to a second aspect of the present disclosure is the medical information processing device according to the first aspect, wherein the processor preferably accepts selection of a second text area from the plurality of superimposed text areas.

[0014] In the medical information processing device according to the third aspect of the present disclosure, in the medical information processing device according to the first or second aspect, it is preferable that the processor determines a text area adjacent to the first text area as the second text area.

[0015] In a medical information processing device according to a fourth aspect of the present disclosure, in a medical information processing device according to any of the first to third aspects, it is preferable that the processor searches a database using text data of the text contained in the integrated text area and extracts multiple candidates with relatively high scores.

[0016] In a medical information processing device according to a fifth aspect of the present disclosure, in the medical information processing device according to the fourth aspect, it is preferable that the processor identifies the item of medical information indicated by the text contained in the integrated text area, and searches a database of the identified item to extract multiple candidates.

[0017] A medical information processing device according to a sixth aspect of the present disclosure is a medical information processing device according to any one of the first to third aspects, wherein the processor preferably extracts multiple candidates by inputting text data of text contained in a text area integrated into a trained model that outputs medical information candidates when text data is input.

[0018] In a medical information processing device according to a seventh aspect of the present disclosure, in a medical information processing device according to any of the first to sixth aspects, it is preferable that the processor determines the likelihood of the text contained in the text area being medical information based on the text data of the text contained in the text area, and superimposes the text area on the image data based on the likelihood of it being medical information.

[0019] A medical information processing device according to an eighth aspect of the present disclosure is preferably a medical information processing device according to any one of the first to seventh aspects, further comprising a camera that captures text indicating medical information and converts it into image data, a display on which a text area is superimposed on the image data, and an input device that accepts selection of a first text area.

[0020] In order to achieve the above object, a medical information processing device according to a ninth aspect of the present disclosure is a medical information processing device that includes at least one processor and at least one memory that stores instructions to be executed by the processor, and the processor acquires image data including text indicating medical information, extracts text areas from the image data, converts the text included in the text areas into text data, and identifies the items of medical information indicated by the text for each text area based on the text data.

[0021] According to this aspect, it is possible to assist in the task of identifying the items to be input and inputting medical information into a medical information processing system.

[0022] A medical information processing device according to a tenth aspect of the present disclosure is the medical information processing device according to the ninth aspect, wherein the items preferably include at least one of a drug name, dosage and administration, and time of administration.

[0023] A medical information processing device according to an eleventh aspect of the present disclosure is preferably a medical information processing device according to the ninth or tenth aspect, further comprising a database in which master medical information for each item is stored, and the processor searches the database with the converted text data to identify the item.

[0024] In a medical information processing device according to a twelfth aspect of the present disclosure, in the medical information processing device according to the ninth or tenth aspect, it is preferable that the processor identifies the items by inputting text data converted into a trained model that outputs items of medical information when text data is input.

[0025] In order to achieve the above-mentioned object, a medical information processing method according to a thirteenth aspect of the present disclosure is a medical information processing method in which at least one processor acquires image data including text indicating medical information, extracts a text area from the image data, converts the text included in the text area into text data, superimposes the text area on the image data, accepts selection of a first text area from multiple superimposed text areas, integrates the selected first text area and a second text area consisting of one or more text areas different from the first text area into a single text area, and displays multiple selectable candidates of medical information indicated by the text in the integrated text area based on the text data of the text included in the integrated text area.

[0026] According to this aspect, it is possible to support the task of correcting recognition errors in OCR processing and inputting medical information into a medical information processing system.

[0027] In order to achieve the above object, a medical information processing method according to a fourteenth aspect of the present disclosure is a medical information processing method in which at least one processor acquires image data including text indicating medical information, extracts text areas from the image data, converts the text included in the text areas into text data, and identifies the items of medical information indicated by the text for each text area based on the text data.

[0028] According to this aspect, it is possible to assist in the task of identifying the items to be input and inputting medical information into a medical information processing system.

[0029] In order to achieve the above object, a program according to a fifteenth aspect of the present disclosure is a program that causes a computer to execute the medical information processing method according to the thirteenth or fourteenth aspect. A non-transitory computer-readable recording medium, such as a CD-ROM (Compact Disk-Read Only Memory), that stores the program according to the fifteenth aspect is also included in the present disclosure. [Effects of the Invention]

[0030] According to the present invention, it is possible to assist in the task of inputting medical information into a medical information processing system. [Brief explanation of the drawings]

[0031] [Figure 1] FIG. 1 is a block diagram showing the electrical configuration of a smartphone. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the medical information processing device. [Figure 3] FIG. 3 is a flowchart showing an example of processing of the medical information processing method. [Figure 4] FIG. 4 is a diagram illustrating an example of the GUI. [Figure 5] FIG. 5 is a diagram illustrating an example of the GUI. [Figure 6] FIG. 6 is a diagram illustrating an example of the GUI. [Figure 7] FIG. 7 is a diagram showing an example of the GUI. [Figure 8] FIG. 8 is a diagram illustrating an example of the GUI. [Figure 9] FIG. 9 is a diagram illustrating an example of the GUI. [Figure 10] FIG. 10 is a block diagram showing the functional configuration of the medical information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0032] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0033] <Overview of medical information processing equipment> The medical information processing device according to the present disclosure is an input support device that supports a user in inputting medical information recorded in a medicine notebook, prescriptions, etc. into a medical information system such as an electronic medical history and an electronic medical record. In particular, the medical information processing device according to the present disclosure automatically or manually selects only character strings to be input into the system from the recognition results of OCR processing, and supports the user in selecting only character strings that exist in various masters as multiple candidates and inputting the selected character string into the medical information system.

[0034] The medical information processing device is mounted on a mobile terminal device, for example. The mobile terminal device includes at least one of a mobile phone, a PHS (Personal Handyphone System), a smartphone, a PDA (Personal Digital Assistant), a tablet computer terminal, a notebook personal computer terminal, and a portable game console. The medical information processing device may also be mounted on a fixed desktop computer, a workstation, or configured in combination with a separate digital camera and display. Below, a medical information processing device configured as a smartphone will be described in detail with reference to the drawings.

[0035] <Smartphone electrical configuration> Fig. 1 is a block diagram showing the electrical configuration of a smartphone 10. As shown in Fig. 1, the smartphone 10 includes a touch panel display 14, a speaker 16, a microphone 18, an in-camera 20, an out-camera 22, a light 24, and a switch 26, as well as a CPU (Central Processing Unit) 28, a wireless communication unit 30, a call unit 32, a memory 34, an external input / output unit 40, a GPS receiver 42, and a power supply unit 44.

[0036] The touch panel display 14 includes a display unit that displays images and the like, and a touch panel unit (an example of an "input device") that is disposed in front of the display unit and accepts touch input. The display unit is, for example, a color LCD (Liquid Crystal Display) panel, and is an example of a display device.

[0037] The touch panel unit (an example of an "input device") is, for example, a capacitive touch panel provided in a planar form on a light-transmitting substrate body, and having light-transmitting position detection electrodes and an insulating layer provided on the position detection electrodes. The touch panel unit generates and outputs two-dimensional position coordinate information corresponding to a user's touch operation. Touch operations include a tap operation, a double tap operation, a flick operation, a swipe operation, a drag operation, a pinch-in operation, and a pinch-out operation.

[0038] The speaker 16 is an audio output unit that outputs audio during a call and when playing back a video. The microphone 18 is an audio input unit that inputs audio during a call and when shooting a video. The in-camera 20 is an imaging device that captures videos and still images. The out-camera 22 is an imaging device that captures videos and still images. The light 24 is a light source that emits illumination light when shooting with the out-camera 22, and is composed of, for example, an LED (Light Emitting Diode).

[0039] The switch 26 is an input member that receives instructions from the user. The switch 26 is a push-button switch that turns on when pressed with a finger or the like and turns off when the finger is released due to the restoring force of a spring or the like.

[0040] The CPU 28 is an example of a processor that executes instructions stored in the memory 34. The CPU 28 operates in accordance with the control program and control data stored in the memory 34, and controls all the components of the smartphone 10. The CPU 28 has a mobile communication control function that controls all the components of the communication system to perform voice communication and data communication via the wireless communication unit 30, and an application processing function.

[0041] The CPU 28 also has an image processing function for displaying moving images, still images, text, etc. on the touch panel display 14. This image processing function visually conveys information such as still images, moving images, and text to the user. The CPU 28 also acquires two-dimensional position coordinate information corresponding to a touch operation by the user from the touch panel portion of the touch panel display 14. The CPU 28 also acquires an input signal from the switch 26.

[0042] The hardware structure of CPU28 is composed of various processors as shown below. ) The various types of processors include CPUs (Central Processing Units), which are general-purpose processors that execute software (programs) and function as various functional units, GPUs (Graphics Processing Units), which are processors specialized for image processing, PLDs (Programmable Logic Devices), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically for executing specific processes.

[0043] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Also, multiple functional units may be configured with a single processor. Examples of multiple functional units configured with a single processor include, first, a configuration in which a single processor is configured with a combination of one or more CPUs and software, as typified by a client or server computer, and this processor operates as multiple functional units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple functional units on a single IC (Integrated Circuit) chip, as typified by an SoC (System On Chip). In this way, the various functional units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0044] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0045] The in-camera 20 and the out-camera 22 capture moving images and still images in accordance with instructions from the CPU 28. The in-camera 20 and the out-camera 22 have the same internal configuration. The in-camera 20 and the out-camera 22 each have a photographing lens, an imaging element, and an image processing unit (not shown).

[0046] The in-camera 20 and the out-camera 22 each receive subject light via a photographing lens using an image sensor. The image sensor is a photoelectric conversion element such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charge-Coupled Device), and has R (red), G (green), and B (blue) color filters (not shown) provided on its light-receiving surface. The subject light forms an image on the light-receiving surface of the image sensor, which converts the subject light imaged on the light-receiving surface into an electrical signal based on the R, G, and B color signals. The image processing unit performs predetermined processing on the analog image signal output from the image sensor to convert it into a digital image signal.

[0047] The in-camera 20 and the out-camera 22 may convert the image data of the captured moving images and still images into compressed image data such as MPEG (Moving Picture Experts Group) or JPEG (Joint Photographic Experts Group).

[0048] The CPU 28 stores the video and still images captured by the in-camera 20 and the out-camera 22 in the memory 34. The CPU 28 may also output the video and still images captured by the in-camera 20 and the out-camera 22 to the outside of the smartphone 10 via the wireless communication unit 30 or the external input / output unit 40.

[0049] Furthermore, the CPU 28 displays the video and still images captured by the in-camera 20 and the out-camera 22 on the touch panel display 14. The CPU 28 may use the video and still images captured by the in-camera 20 and the out-camera 22 in application software.

[0050] The CPU 28 may illuminate the subject with fill light by turning on the light 24 when capturing an image with the outer camera 22. The light 24 may be turned on and off by a touch operation on the touch panel display 14 or by operating the switch 26 by the user.

[0051] The wireless communication unit 30 performs wireless communication with a base station device included in the mobile communication network in accordance with instructions from the CPU 28. Using this wireless communication, the smartphone 10 transmits and receives various file data such as audio data and image data, e-mail data, and the like, and receives Web (abbreviation for World Wide Web) data, streaming data, and the like.

[0052] The speaker 16 and the microphone 18 are connected to the communication unit 32. The communication unit 32 decodes the audio data received by the wireless communication unit 30 and outputs it from the speaker 16. The communication unit 32 converts the user's voice input through the microphone 18 into audio data that can be processed by the CPU 28 and outputs it to the CPU 28.

[0053] The memory 34 stores instructions to be executed by the CPU 28. The memory 34 is composed of an internal storage unit 36 ​​built into the smartphone 10 and an external storage unit 38 that is detachable from the smartphone 10. The internal storage unit 36 ​​and the external storage unit 38 are realized using known storage media. The external storage unit 38 may be a non-transitory computer-readable recording medium.

[0054] The memory 34 stores the control program of the CPU 28, control data, application software, address data associated with names and telephone numbers of communication partners, data of emails sent and received, web data downloaded by web browsing, downloaded content data, etc. The memory 34 may also temporarily store streaming data, etc.

[0055] The external input / output unit 40 serves as an interface with external devices connected to the smartphone 10. The smartphone 10 is directly or indirectly connected to other external devices by communication or the like via the external input / output unit 40. The external input / output unit 40 transmits data received from external devices to each component within the smartphone 10, and also transmits data within the smartphone 10 to external devices.

[0056] Examples of communication means include a Universal Serial Bus (USB), IEEE (Institute of Electrical and Electronics Engineers) 1394, the Internet, a wireless local area network (LAN), Bluetooth (registered trademark), RFID (Radio Frequency Identification), and infrared communication. Examples of external devices include a headset, an external charger, a data port, an audio device, a video device, a smartphone, a PDA, a personal computer, and earphones.

[0057] The GPS receiver 42 detects the position of the smartphone 10 based on positioning information from GPS satellites ST1, ST2, . . . , STn.

[0058] The power supply unit 44 is a power supply source that supplies power to each unit of the smartphone 10 via a power supply circuit (not shown). The power supply unit 44 includes a lithium ion secondary battery. The power supply unit 44 may also include an A / D conversion unit that generates a DC voltage from an external AC power supply.

[0059] The smartphone 10 configured in this manner is set to a shooting mode in response to a user's instruction input using the touch panel display 14 or the like, and can capture moving images and still images using the in-camera 20 and the out-camera 22.

[0060] When the smartphone 10 is set to the shooting mode, it enters a shooting standby state, and a video is captured by the in-camera 20 or the out-camera 22, and the captured video is displayed on the touch panel display 14 as a live view image.

[0061] The user can visually check the live view image displayed on the touch panel display 14 to determine the composition, confirm the subject they want to photograph, and set the photographing conditions.

[0062] When the smartphone 10 is in a standby state for shooting and receives an instruction to shoot by a user using the touch panel display 14 or the like, the smartphone 10 performs AF (Autofocus) and (Auto Exposure) control, and shoots and stores videos and still images.

[0063] First Embodiment [Functional configuration of medical information processing device] 2 is a block diagram showing the functional configuration of the medical information processing device 100 implemented by the smartphone 10. As shown in FIG. 2, the medical information processing device 100 includes a medical information master database 102, an image acquisition unit 104, a character recognition unit 106, a selection unit 108, a superimposed display unit 110, an integration unit 112, a candidate output unit 114, and a determination unit 116. The functions of the medical information master database 102 are embodied by the memory 34. The functions of the image acquisition unit 104, the character recognition unit 106, the selection unit 108, the superimposed display unit 110, the integration unit 112, the candidate output unit 114, and the determination unit 116 are embodied by the CPU 28 executing a program stored in the memory 34.

[0064] The medical information master database 102 is a database that stores master data for each item of medical information. The items of medical information include at least one of the following: drug name, dosage and administration, and time of administration. The medical information master database 102 includes a drug name database 102A, a dosage and administration database 102B, and a time of administration database 102C.

[0065] The drug name database 102A is a database that stores master data for the names of each drug. The drug name includes at least one of the following: "generic name," "product name," "type," "use," and "manufacturer name."

[0066] The dosage and administration database 102B is a database that stores master data on dosage and administration for each drug. The dosage of a drug includes the "number of times to take it per day" and the "time to take it", etc. The dosage of a drug includes the "amount to take at one time" and the "amount to take per day", etc.

[0067] The dosing time database 102C is a database that stores a master of the dosing time for each medicine. The dosing time refers to the timing at which a medicine is taken. The dosing time includes "before meals," "during meals," "after meals," and "between meals," based on "breakfast," "lunch," and "dinner." The dosing time also includes cases where "breakfast," "lunch," and "dinner" are not distinguished, such as "after every meal." The dosing time includes times based on sleep, such as "before going to bed" and "upon waking up."

[0068] The image acquisition unit 104 acquires an image (an example of "image data") including a character string (an example of "text") indicating medical information. For example, the image acquisition unit 104 acquires a captured image of an entire paper medium, such as a medicine notebook or prescription, which contains multiple pieces of medical information such as the name of a drug, dosage, and when to take it, captured by the in-camera 20 or the out-camera 22. The image acquisition unit 104 may acquire a captured image of a portion of the paper medium, or may acquire a captured image of medical information displayed on a display. The image acquisition unit 104 may acquire an image from another device via the wireless communication unit 30, the external storage unit 38, or the external input / output unit 40.

[0069] The character recognition unit 106 extracts character string areas (an example of a "text area") from an image, and converts the character strings contained in the character string areas into text data by character recognition using OCR processing on the extracted character string areas, thereby acquiring a set of character information. Characters include kanji, hiragana, katakana, alphabets, numbers, and various symbols. A character string is one or more characters arranged consecutively. A character string area is an area of ​​an image where a character string exists.

[0070] The character recognition unit 106 may perform preprocessing on the recognized character string. The preprocessing may be a process of deleting spaces. The preprocessing may be a process of deleting characters that are not used in medical information, for example, a process of deleting characters that are not used in drug names. Characters that are not used in drug names include, for example, circled numbers and black parentheses. If all master character strings stored in the medical information master database 102 are registered in full-width characters, the preprocessing may be a process of converting half-width characters to full-width characters.

[0071] The character recognition unit 106 may perform error recovery using AI (Artificial Intelligence) that has been trained in advance to output the error-recovered drug name, dosage name, and time of administration when text data is input, and update the OCR-processed text data.

[0072] The selection unit 108 selects, from a collection of character information, character strings of medical information to be input to the medical information system and noise information that is not to be input. The selection unit 108 determines the likelihood of each piece of text data in each character string region being medical information. Here, the selection unit 108 calculates a score for the text data in each character string region in terms of likelihood that the text data is an item of medical information, and selects character string regions with relatively high scores as medical information.

[0073] The selection unit 108 includes a selection AI 108A. The selection AI 108A is a trained model that has been trained in advance to output scores for each item, such as "likely a drug name," "likely a dosage and administration," and "likely a time point of administration," indicated by the text data when the text data is input. The selection unit 108 inputs a set of character information into the selection AI 108A for each character string, and calculates each score for each item. The training of the selection AI 108A may be performed by a computer different from the medical information processing device 100. Parameters such as weights of the trained selection AI 108A may be stored in the memory 34.

[0074] To train the selection AI 108A that outputs a score of "likely a drug name," it can be trained to output 1 when an actual drug name is input, and 0 when a non-actual drug name is input. The same applies to the selection AI 108A that outputs a score of "likely a dosage and administration" and the selection AI 108A that outputs a score of "likely a time point of administration."

[0075] The selection unit 108 may compare the character string of the text data with the character string contained in the medical information master database 102, and if the character string of the text data is contained in the medical information master database 102, may give the character string a relatively high score for being an item of medical information.

[0076] The superimposed display unit 110 displays the image acquired by the image acquisition unit 104 on the touch panel display 14. The superimposed display unit 110 also displays the character string area extracted by the character recognition unit 106 superimposed on the image. For example, the superimposed display unit 110 displays a graphic representing the character string area superimposed on the image. The graphic representing the character string area is, for example, a frame surrounding the character string area. The graphic representing the character string area may be an underline or a rectangle that fills the character string area.

[0077] The superimposed display unit 110 may superimpose a character string region on an image based on the likelihood that a character string included in the character string region is an item of medical information calculated by the selection unit 108. For example, the superimposed display unit 110 may superimpose a graphic representing a character string region on an image only for character string regions having a relatively high score of likelihood of being an item of medical information, and may not superimpose a graphic representing a character string region on a character string region having a relatively low score.

[0078] The superimposed display unit 110 may change the color of the character string region or the color of the figure indicating the character string region depending on the scores of "likelihood of a drug name," "likelihood of dosage and administration," and "likelihood of time of administration." For example, the superimposed display unit 110 may color-code the frame such that a character string region with a relatively high score of "likelihood of a drug name" is "red," a character string region with a relatively high score of "likelihood of dosage and administration" is "yellow," a character string region with a relatively high score of "likelihood of time of administration" is "green," etc.

[0079] The superimposed display unit 110 may vary the density of the character string region or the density of the figure representing the character string region depending on the accuracy of each of the scores for "likelihood of a drug name," "likelihood of dosage and administration," and "likelihood of time of administration." For example, the superimposed display unit 110 may increase the density of the color of the frame as the accuracy of the score becomes relatively higher. The superimposed display unit 110 may display a rectangle in the character string region according to the accuracy of the score, or may display a bar in the character string region with a length according to the accuracy of the score.

[0080] The user may select a frame surrounding the character string area displayed by the superimposed display unit 110, and select "drug name," "dosage and administration," "time of administration," etc. from the character string in the selected frame. The selection unit 108 may acquire the result of manual selection by the user.

[0081] The integrating unit 112 accepts a user selection of a first character string area (an example of a "first text area") that is one of the multiple character string areas displayed in a superimposed manner. The user selects the first character string area using the touch panel portion of the touch panel display 14. The integrating unit 112 may automatically select the first character string area in order from the top or from the left, regardless of the user selection.

[0082] Furthermore, the integrating unit 112 integrates the selected first character string region and a second character string region (an example of a "second text region") different from the first character string region into one character string region. The second character string region is composed of one or more character string regions that do not include the first character string region. For example, the integrating unit 112 accepts one or more character string regions from among the multiple character string regions displayed in a superimposed manner, and determines the accepted one or more character string regions as the second character string region. The integrating unit 112 may also select a character string region adjacent to the first character string region as the second character string region. The first character string region and the second character string region may be a single character string region that has been divided into multiple character strings by a line break or the like.

[0083] The candidate output unit 114 extracts candidates for medical information indicated by the character strings included in the integrated character string area based on the text data resulting from the OCR processing of the character strings included in the integrated character string area. Here, the candidate output unit 114 extracts multiple candidates and displays them on the touch panel display 14 so that they can be selected.

[0084] The candidate output unit 114 performs a partial match search in the medical information master database 102 using the character string of the text data as input. If a partially matching character string exists in the medical information master database 102, the candidate output unit 114 presents the record as a candidate. If a partially matching character string does not exist in the medical information master database 102, the candidate output unit 114 calculates the distance between the character string of the text data and the character strings of all records in the medical information master database 102, and presents records with relatively close distances to the user as multiple candidates. The distance is, for example, the Levenshtein distance. The candidate output may be based only on the result of the partial match search, or may be based only on the distance result without performing the partial match search process.

[0085] In this way, the candidate output unit 114 obtains a score by comparing the text data with the records in the medical information master database 102 in a round-robin manner, and displays, as multiple candidates, records with relatively high scores among the records included in the medical information master database 102. The candidate output unit 114 may display a certain number of records with relatively high scores as multiple candidates, or may display all records that are a perfect match or a partial match in a search of the medical information master database 102 as multiple candidates. The candidate output unit 114 may also scroll through the multiple candidates.

[0086] The candidate output unit 114 may obtain a score using a database for the item identified by the selection unit 108 from among the drug name database 102A, the dosage and administration database 102B, and the dosing time database 102C of the medical information master database 102. The candidate output unit 114 may also identify a database from the selection result of the selection unit 108. For example, the candidate output unit 114 obtains a score using the drug name database 102A for text data in a character string region for which the selection unit 108 has a relatively high score for "drug name likeness."

[0087] The candidate output unit 114 may specify a database based on a user input. For example, when a user touches and holds a character string area on the touch panel display 14 that the user wants to select for a certain period of time or longer, the candidate output unit 114 may display a menu such as "Match as drug name" or "Match as dosage and administration." When the user selects one of the options from the displayed menu, the candidate output unit 114 may select the corresponding database. In addition, when only the drug name is to be input to the medical information system, the drug name database 102A may be specified directly without displaying a menu or the like.

[0088] The confirmation unit 116 confirms the correct medical information from among multiple candidates of medical information indicated by the character string in the integrated character string area. For example, the confirmation unit 116 confirms the candidate selected by the user from among the multiple candidates displayed on the touch panel display 14 as the correct medical information. The user compares the character string in the captured image with the multiple proposed candidates and selects the correct candidate. The selected candidate may be input into the medical information system as text data corresponding to the character string area. Alternatively, the information to be input into the medical information system may be determined based on a key linked to the confirmed medical information.

[0089] [Medical information processing method] Fig. 3 is a flowchart showing an example of processing of a medical information processing method using the medical information processing device 100. Fig. 4, Fig. 5, and Fig. 6 are diagrams showing examples of GUIs (Graphical User Interfaces) in each step of the medical information processing method. The medical information processing method is realized by the CPU 28 executing a medical information processing program stored in the memory 34.

[0090] When the medical information processing method is executed, the smartphone 10 transitions to a photographing mode. In the photographing mode, a photographing button B1 (see FIG. 4), which is a UI (User Interface) button that the user can tap, is displayed on the touch panel display 14. When the user taps the photographing button B1, photographing is performed by the outer camera 22. In step S1, the user photographs medical information that the user wishes to input to the medical information system with the outer camera 22. The image acquisition unit 104 acquires the photographed image and stores it in the memory 34. The superimposed display unit 110 displays the photographed image on the touch panel display 14.

[0091] F4A in Fig. 4 is a diagram showing an example of the display on touch panel display 14 in step S1. In the example shown in F4A, image IM1 is displayed. Image IM1 is a captured image of a page of a medicine notebook listing prescribed medicines.

[0092] After the image capture in step S1 is performed, the smartphone 10 transitions to character recognition mode. In the character recognition mode, an OCR button B2 (see FIG. 4), which is a UI button that the user can tap, is displayed on the touch panel display 14. When the user taps the OCR button B2, the processing in step S2 is performed. Note that the OCR button B2 is not essential to the GUI, and instead of tapping the OCR button B2, the OCR processing may be triggered by the successful acquisition of the captured image. Furthermore, in cases where the results of the OCR processing are unsatisfactory, a re-capture button that executes re-capture may be provided on the GUI.

[0093] In step S2, the character recognition unit 106 extracts a character string area from the captured image, and converts the character string included in the character string area into text data by OCR processing.

[0094] In step S3, the superimposed display unit 110 superimposes the character string area extracted by the character recognition unit 106 on the image IM1 displayed on the touch panel display 14. F4B in FIG. 4 is a diagram showing an example of the display on the touch panel display 14 in step S3. In the example shown in F4B, an image IM2 is displayed. The image IM2 is an image in which a frame FA is superimposed on the image IM1. The frame FA is a dashed rectangle that indicates the outline of the character string area.

[0095] In step S4, the integrating unit 112 accepts a user selection of a first character string area, which is one of the multiple character string areas displayed in a superimposed manner. The user selects the character string area of ​​medical information that the user wishes to input to the medical information system using the touch panel unit of the touch panel display 14. The integrating unit 112 may make the appearance of a graphic representing the selected first character string area different from the appearance of a graphic representing an unselected character string area.

[0096] F5A in FIG. 5 is a diagram showing an example of the display on the touch panel display 14 in step S4. In the example shown in F5A, image IM3 is displayed. Image IM3 is an image in which frame FA and frame FS1 are superimposed on image IM1. Frame FS1 is a solid-line rectangle indicating the first character string area selected by the user. In this example, "AIUEO TABLETS 20 mg" is the first character string area.

[0097] In step S5, the integrating unit 112 integrates the first character string region and a second character string region that is composed of one or more character string regions different from the first character string region into a single character string region. Here, an example in which the second character string region does not exist will be described. In other words, the first character string region is treated as the integrated character string region.

[0098] In step S6, the candidate output unit 114 displays, on the touch panel display 14, a plurality of selectable candidates for medical information indicated by the text data of the character strings included in the integrated character string area.

[0099] F5B is text data converted in step S2 by the character recognition unit 106 through character recognition of the character string contained in the first character string area, "AIUEO TABLETS 20mg." In the example shown in F5B, "U" was erroneously recognized as "K" in the OCR process, resulting in recognition as "AIUEO TABLETS 20mg."

[0100] The candidate output unit 114 obtains a score by comparing "AIUEO TABLETS 20mg" in the text data of the integrated character string region with character strings included in the medical information master database 102 in a round-robin manner. Here, the score of "AIUEO TABLETS 20mg" included in the medicine name database 102A is relatively high.

[0101] In the example shown in F5A, a candidate list L1 containing multiple candidates for the character string in the integrated character string region is displayed. The five candidates in candidate list L1 are arranged in descending order of their relative scores. "Aiueo Tablets 20mg" is displayed at the top of candidate list L1.

[0102] Candidate list L1 may include six or more candidates. In the example shown in F5A, there is only space to display five candidates, but candidates with scores sixth and above may be displayed by the user's scrolling operation. Increasing the space for displaying the candidate list reduces the space for displaying image IM3, but scrolling the list allows a relatively larger number of candidates to be displayed while maintaining the display size of image IM3.

[0103] The candidate list L1 may display the Levenshtein distance of each candidate to the recognized text data.

[0104] In step S7, the user selects a correct character string from among the multiple candidates in the candidate list L1. The user can select the correct character string by tapping on the touch panel display 14. The confirmation unit 116 confirms the candidate selected by the user as the correct medical information. Here, for example, by selecting "AIUEO TABLETS 20mg" at the top of the candidate list L1, the text data of "AIUEO TABLETS 20mg" is entered into the medical information system. A confirmation button may be separately provided on the GUI, and the confirmation process may be executed by tapping the confirmation button.

[0105] The user repeats the above process for all text areas included in the image IM1 that should be input to the medical information system. The determination unit 116 may fill in the frame FA indicating the character string area for which input to the medical information system has been completed, so that it can be distinguished from an uninputted character string area.

[0106] Note that a character string representing one piece of medical information may be recognized as two or more separate character string regions. For example, in the example shown in F4B of FIG. 4, for the name of the ninth listed drug, "(Post) DAZHIDZUDEDOB TABLETS 5mg 'TACHITS'," the first half "(Post) DAZHIDZUDEDOB TABLETS 5mg" and the second half "'TACHITS'" are recognized as different character string regions. Even in such a case, the medical information processing device 100 can integrate the character string regions as follows and output the correct medical information candidate.

[0107] In step S4, it is assumed that the user selects the first half, "(Post) DAZHZUDEDOBA TABLETS 5mg," as the first character string region. F6A in FIG. 6 is a diagram showing an example of the display on the touch panel display 14 in this case. In the example shown in F6A, image IM4 is displayed. Image IM4 is an image in which frames FA and FS2 are superimposed on image IM1. Frame FS2 is a solid-line rectangle indicating "(Post) DAZHZUDEDOBA TABLETS 5mg," which is the first character string region selected by the user. The text data, which is the character string recognized in this rectangular portion by OCR processing, is "(Post) DAZHZUDEDOBA TABLETS 5mg," and the correct name of the drug is "DAZHZUDEDOBA TABLETS 5mg 'Tachitsu'."

[0108] Furthermore, as shown in F6A, a candidate list L2 including multiple candidates for "(Post) Dajizudedoba Tablets 5mg" is displayed on the touch panel display 14. Here, candidate acquisition processing is performed based on the text data "(Post) Dajizudedoba Tablets 5mg", but as a result of partial match search and candidate acquisition based on distance in the candidate output unit 114, there is a large discrepancy between this text data and the correct answer, "Dajzudedoba Tablets 5mg 'Tachitsu'", and the correct drug name, "Dajzudedoba Tablets 5mg 'Tachitsu'", is not included in the candidate list L2.

[0109] In such a case, the user can select multiple character string areas and have them recognized as a single character string area. That is, in step S4, the integration unit 112 accepts a first character string area and one or more second character string areas different from the first character string area from the multiple character string areas displayed in a superimposed manner. In this case, the user can select the latter part, "'tachitsu'" as the second character string area. The text data, which is the character string recognized in this rectangular portion by OCR processing, is "'tachitsu'".

[0110] F6B in Fig. 6 is a diagram showing an example of the display on touch panel display 14 in this case. In the example shown in F6B, image IM5 is displayed. Image IM5 is an image in which frame FA, frame FS2, and frame FS3 are superimposed. Frame FS3 is a solid-line rectangle indicating the second character string area "Tachitsu" selected by the user.

[0111] Preferably, character string areas that the user can select as the second character string area are limited to only character string areas adjacent to the first character string area. When the first character string area is selected, the superimposed display unit 110 may display character string areas that are not adjacent to the first character string area in an inactive gray state, indicating that they are not selectable, and display character string areas that are adjacent to the first character string area in an active state, indicating that only the character string areas are selectable.

[0112] In step S5, the integrating unit 112 integrates the first character string region and the second character string region into one character string region. As a result, "(Go) DAZHIDZUDEDOBA TABLETS 5MG" and "'TACHITS'" are integrated to become "(Go) DAZHIDZUDEDOBA TABLETS 5MG 'TACHITS'". In step S6, the candidate output unit 114 displays, on the touch panel display 14, a plurality of selectable candidates for medical information indicated by the character strings in the integrated character string region.

[0113] In the example shown in F6B, a candidate list L3 containing multiple candidates for the character string "(after) DAZHIDZUDEDOBAT 5MG 'TACHITS'" in the integrated character string area is displayed. The five candidates in candidate list L3 are arranged from the top in descending order of their scores. The correct answer, the name of the drug "DAZHIDZUDEDOBAT 5MG 'TACHITS'", is displayed at the top of candidate list L3.

[0114] In prescriptions and the like, the character string "(after)" may be added before the official trade name of a drug to clearly indicate that it is a generic drug, but in the process of matching drug names, the character string "(after)" becomes noise. The character string "(after)" may be deleted in the process of deleting characters not used in drug names in the character recognition unit 106.

[0115] In step S7, the user selects the correct character string from among the multiple candidates in candidate list L3. In this example, the user selects "Dajizudedoba Tablets 5mg 'Tachitsu'" from candidate list L3.

[0116] As described above, according to the first embodiment, even if a character string representing one piece of medical information is extracted as multiple character string areas, the misrecognition can be corrected and the extracted character string areas can be integrated into one character string area, thereby improving the matching accuracy and facilitating the input work into the medical information system.

[0117] Furthermore, according to the first embodiment, the item of medical information indicated by the character string contained in the character string area is identified based on the text data. Therefore, by displaying the character string area according to the item of medical information, the user can recognize the item of medical information, and the task of inputting medical information into the medical information processing system can be assisted.

[0118] Furthermore, according to the first embodiment, the item of medical information indicated by the character string contained in the character string area is identified based on the text data. Therefore, by searching the database for the identified item and extracting multiple candidates, unnecessary database searches can be prevented, search time can be shortened, and the task of inputting medical information into the medical information processing system can be supported.

[0119] Here, for the name of a pharmaceutical product, "(Post) DAZHZUDEDOB TABLETS 5mg 'TACHITS'," the first half "(Post) DAZHZUDEDOB TABLETS 5mg" and the second half "'TACHITS'" are recognized as different character string regions, and the first half "(Post) DAZHZUDEDOB TABLETS 5mg" is selected as the first character string region and the second half "'TACHITS'" is selected as the second character string region. However, even if the second half "'TACHITS'" is selected as the first character string region and the second half "(Post) DAZHZUDEDOB TABLETS 5mg" is selected as the second character string region, the character string regions integrated by the integrating unit 112 will be the same, and the same effect can be achieved. In other words, when a character string representing one piece of medical information is extracted as multiple character string regions, the misrecognition can be corrected and integrated into a single character string region regardless of the order of selection.

[0120] <Second embodiment> The image acquired by the image acquisition unit 104 in step S1 is not limited to a captured image on a paper medium, but may be an image of medical information displayed on the display of another device, etc. Furthermore, the character string areas integrated by the integration unit 112 in step S5 are not limited to character string areas arranged horizontally, but may be character string areas arranged vertically. The processing of the medical information processing method according to the second embodiment is the same as the flowchart shown in Fig. 3. Figs. 7 and 8 are diagrams showing an example of a GUI in the second embodiment.

[0121] F7A in Fig. 7 is a diagram showing an example of the display on the touch panel display 14 in step S1. In the example shown in F7A, an image IM11 is displayed. Image IM11 is an image of an electronic medication notebook displayed on the display of a smartphone (not shown) owned by a patient or the like, captured by the external camera 22. The electronic medication notebook is an application program for managing information similar to that of a paper medication notebook on a smartphone or the like.

[0122] F7B in FIG. 7 is a diagram showing an example of the display on the touch panel display 14 in step S3. In the example shown in F7B, image IM12 is displayed. Image IM12 is an image in which a dashed rectangular frame FA indicating the outline of a character string area is superimposed on image IM11. In the example shown in F7B, the name of the drug, "Kikukecosa Sheath Hydrochloride GH Tablets 0.375mgIJ "KLMN"," is written with the first half "Kikukecosa Sheath Hydrochloride GH Tablets 0.3" and the second half "75mgIJ "KLMN"" separated into upper and lower lines, and each is recognized as a different character string area.

[0123] F8A in FIG. 8 is a diagram showing an example of the display on the touch panel display 14 when the first character string region is selected in step S4. In the example shown in F8A, image IM13 is displayed. Image IM13 is an image in which frame FA and frame FS11 are superimposed on image IM11. Frame FS11 is a solid-line rectangle indicating the first character string region selected by the user, "Kikukecosa Sheath Hydrochloride GH Tablet 0.3." The text data, which is the character string recognized in this rectangular portion by OCR processing, is "Kikukecosa Sheath Hydrochloride GH Tablet 0.3," and the correct drug name is "Kikukecosa Sheath Hydrochloride GH Tablet 0.375 mg IJ "KLMN."

[0124] Furthermore, as shown in F8A, a candidate list L11 including multiple candidates for "Kikukecosa Sheath Hydrochloride GH Tablets 0.3" is displayed on the touch panel display 14. Here, candidate acquisition processing is performed based on the text data "Kikukecosa Sheath Hydrochloride GH Tablets 0.3," but as a result of partial match search and candidate acquisition based on distance in the candidate output unit 114, there is a large discrepancy between this text data and the correct answer, "Kikukecosa Sheath Hydrochloride GH Tablets 0.375 mg IJ "KLMN"," and the correct drug name, "Kikukecosa Sheath Hydrochloride GH Tablets 0.375 mg IJ "KLMN"," is not included in the candidate list L11.

[0125] In this case, the user simply selects the character string region "75mgIJ "KLMN"" adjacent to the bottom of the first character string region "Kikukecosa Sheath Hydrochloride GH Tablets 0.3". In step S4, the integrating unit 112 accepts the selected character string region "75mgIJ "KLMN"" as the second character string region.

[0126] F8B in FIG. 8 is a diagram showing an example of the display on touch panel display 14 in this case. In the example shown in F8B, image IM14 is displayed. Image IM14 is an image in which frame FA, frame FS11, and frame FS12 are superimposed on image IM11. Frame FS12 is a solid-line rectangle that indicates the second character string area selected by the user, "75mgIJ"KLMN"." The text data, which is the character string recognized in this rectangular portion by OCR processing, is "75mgIJ"KLMN"."

[0127] In step S5, the integrating unit 112 integrates the first character string region and the second character string region into one character string region. As a result, "Kikukecosa Sheath Hydrochloride GH Tablet 0.3" and "75mgIJ "KLMN"" are integrated into "Kikukecosa Sheath Hydrochloride GH Tablet 0.375mgIJ "KLMN"".

[0128] In step S6, the candidate output unit 114 displays, on the touch panel display 14, a plurality of selectable candidates for medical information indicated by the character strings in the integrated character string region.

[0129] In the example shown in F8B, a candidate list L12 containing multiple candidates for the character string "Kikukecosa Sheath Hydrochloride GH Tablets 0.375mgIJ "KLMN"" in the integrated character string region is displayed. The correct drug name, "Kikukecosa Sheath Hydrochloride GH Tablets 0.375mgIJ "KLMN"," is displayed at the top of candidate list L3.

[0130] In step S7, the user selects the correct character string from among the multiple candidates in the candidate list L12. In this example, the user selects "Kikukecosa Sheath Hydrochloride GH Tablets 0.375 mg IJ "KLMN"" from the candidate list L12.

[0131] Since electronic medication notebooks are displayed on smartphone displays, which have relatively small screens, character string regions are easily split vertically. In contrast, according to the second embodiment, even if a character string representing one piece of medical information is split vertically and extracted as multiple character string regions, it can be integrated into a single character string region, thereby improving matching accuracy and facilitating the work of inputting data into a medical information system.

[0132] Up to this point, an example has been described in which one character string area is selected as the second character string area. However, the user may select multiple text areas as the second character string area. The integrating unit 112 sets the selected multiple text areas as the second character string area and integrates the first character string area and the second character string area into one character string area. In this case, the integrating unit 112 integrates the selected character string areas in order from the top left to the bottom right. As a result, even if a character string representing one piece of medical information is recognized as three or more separate character string areas, the character string areas can be integrated and the correct medical information candidate can be output. As with the first embodiment, the same effect can be achieved for three or more separate character string areas regardless of the order in which they are selected by the user.

[0133] <Third embodiment> Information that is highly necessary to input from the medication notebook into the medical information system includes the name of the drug, dosage, dosage, daily dosage, and single dosage.

[0134] Among these, the dosage instructions often include numbers, such as "one tablet per day" or "three tablets per day." A correction GUI for the numerical portion may be provided as a recovery method when the character recognition unit 106 erroneously recognizes a numerical value. For example, when the user taps the numerical portion, a drum roll-shaped picker for selecting a numerical value is displayed, and the correction GUI may allow the user to select a numerical value.

[0135] Input items such as dosage and administration that are difficult to master may be input using template input or an alternative GUI for manual input.

[0136] After the name of the drug is determined, the dosage form may be identified by referring to a drug information database (not shown) and appropriate dosage and administration candidates may be displayed. The dosage form is the shape of the drug, such as a tablet, powder, capsule, or injection.

[0137] 9 is a diagram showing an example of a GUI when the determination unit 116 determines the name of a drug as the correct medical information in step S7. Here, the dosage form of the determined drug is shown as a tablet. As shown in FIG. 9, a text box TB and a pull-down menu PM are displayed on the touch panel display 14.

[0138] The text box TB is for the user to input the dosage to be taken at one time. When the text box TB is tapped, a software keyboard (not shown) is displayed on the touch panel display 14. The user can input a numerical value into the text box TB by operating the software keyboard.

[0139] Instead of the text box TB, a pull-down menu may be used to allow the user to select the dose to be taken at one time.

[0140] The pull-down menu PM allows the user to select the unit of the dosage to be taken at one time. In this example, the pull-down menu PM includes the options "tablets," "g," "pieces," and "other." The user can determine the unit by tapping one of the options in the pull-down menu PM.

[0141] For drug names, a master exists in the drug name database 102A, so quantitative evaluation based on distance, etc. is possible. On the other hand, it is difficult to create a database for dosage and administration, which is free format, and it is difficult to quantitatively evaluate using distance. According to the third embodiment, even for dosage and administration that includes numbers, matching accuracy can be improved, and the work of inputting data into the medical information system can be supported.

[0142] <Fourth embodiment> In the first and second embodiments, the user selects the second character string area, but the medical information processing apparatus 100 may automatically select the second character string area.

[0143] For example, the candidate output unit 114 may integrate only the first character string region selected by the user into a character string region, and may perform a brute force search to determine the first Levenshtein distance between the text data of the character string and the master. In the example shown in F6A of Fig. 6, the candidate output unit 114 may perform a brute force search to determine the Levenshtein distance between the text data of the character string "(Go) DAZHIDZUDEDOBA TABLETS 5MG" which is the first character string region, and each record included in the drug name database 102A.

[0144] Next, if there is a character string area horizontally adjacent to the first character string area selected by the user, the integrating unit 112 sets the adjacent character string area as a second character string area. Then, the integrating unit 112 integrates the first character string area and the second character string area into a single character string area. In the example shown in F6A of FIG. 6, the character string area "'Tachitsu'" adjacent to the right of the first character string area "(Go) DAZHIZUDEDOVA TABLETS 5MG" is set as the second character string area.

[0145] The candidate output unit 114 calculates the second Levenshtein distance between the text data of the character string included in the integrated character string area and the master by a brute force method. In the example shown in F6A in Fig. 6, the Levenshtein distance between the text data of the character string "(Go) DAZHIDZUDEDOVA TABLETS 5mg 'TACHITS'" which is the integrated character string area and each record included in the drug name database 102A is calculated by a brute force method.

[0146] The candidate output unit 114 compares the first Levenshtein distance and the second Levenshtein distance thus calculated, and extracts a plurality of candidates having relatively small Levenshtein distances. The candidate output unit 114 obtains, for example, 10 candidates and displays them on the touch panel display 14.

[0147] As described above, according to the fourth embodiment, character string areas adjacent to a selected character string area can be automatically merged, thereby reducing the user's workload.

[0148] Here, we have explained the case where there is one character string area adjacent to the first character string area, but if there are multiple adjacent character string areas, the Levenshtein distance for each combination of character string areas can be calculated in a brute-force manner and similar processing can be performed.

[0149] Although the case where there is a character string area adjacent to the first character string area in the horizontal direction has been described above, the same processing can also be performed when there is a character string area adjacent to the first character string area in the vertical direction. When there are character string areas adjacent to the first character string area in the horizontal direction and character string areas adjacent to the first character string area in the vertical direction, these can be combined.

[0150] <Fifth embodiment> The candidate output unit 114 extracts multiple candidates of medical information by searching the medical information master database 102, but multiple candidates may also be extracted using a trained model.

[0151] FIG. 10 is a block diagram showing the functional configuration of a medical information processing apparatus 100 according to the fifth embodiment. As shown in FIG. 10, the candidate output unit 114 includes a candidate output AI 114A. The candidate output AI 114A is a trained model that has been trained in advance so that, when text data is input, it outputs candidates for medical information indicated by the text data. The candidate output unit 114 inputs text data of character strings included in the integrated character string area to the candidate output AI 114A to obtain multiple candidates. The candidate output AI 114A may be trained on a computer different from the medical information processing apparatus 100.

[0152] According to the fifth embodiment, the matching accuracy can be improved by using a trained model, and the work of inputting data into a medical information system can be assisted.

[0153] <Other> Although an example in which all functions of the medical information processing device 100 are executed by the smartphone 10 has been described here, the medical information processing device 100 may be realized by multiple devices. For example, at least one function of the image acquisition unit 104, character recognition unit 106, selection unit 108, superimposition display unit 110, integrating unit 112, candidate output unit 114, and determination unit 116 may be executed by another device such as a server. Also, the medical information master database 102 may be provided in a server or the like.

[0154] The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention. [Explanation of symbols]

[0155] 10. Smartphone 14...Touch panel display 16...Speaker 18...Microphone 20...In-camera 22...Outside camera 24...Light 26...Switch 28...CPU 30...Radio communication section 32...Telephone section 34...Memory 36...Internal storage 38...External memory unit 40...External input / output section 42...GPS receiver 44...Power supply section 100...Medical information processing device 102...Medical Information Master Database 102A...Drug Name Database 102B...Dosage and Administration Database 102C...Database at the time of administration 104...Image acquisition unit 106...Character recognition section 108...Sorting Department 108A…Selection AI 110...Superimposed display section 112…Integration Department 114...Candidate output section 114A…Candidate output AI 116...Determined part B1...Shooting button B2...OCR button FA...frame FS1...frame FS2...frame FS3...frame FS11…frame FS12…frame IM1...Image IM2...Image IM3...Image IM4...Image IM5...Image IM11...Image IM12...Image IM13...Image IM14...Image L1...Candidate list L2…Candidate list L3…Candidate list L11…Candidate list L12…Candidate list PM...Pull-down menu ST1…GPS satellite ST2…GPS satellite S1 to S7: Steps in the medical information processing method TB...Text box

Claims

1. at least one processor; at least one memory storing instructions for execution by said processor; Equipped with The processor: acquiring image data including text indicating medical information; Extracting text regions from the image data; converting the text contained in the text area into text data; The text area is displayed superimposed on the image data; accepting a selection of a first text area from the plurality of superimposed text areas; Integrating the selected first text area and a second text area that is composed of one or more of the text areas different from the first text area into one text area; displaying a plurality of selectable candidates of medical information indicated by the text in the integrated text area based on text data of the text included in the integrated text area; Medical information processing equipment.

2. The processor: accepting a selection of the second text area from the plurality of text areas displayed in the superimposed manner; The medical information processing device according to claim 1 .

3. The processor: determining a text area adjacent to the first text area as the second text area; The medical information processing device according to claim 1 .

4. The processor: searching a database using text data of the text included in the integrated text region to extract the plurality of candidates having relatively high scores; The medical information processing device according to claim 1 .

5. The processor: identifying an item of medical information represented by text contained in the consolidated text region; searching a database of the identified items to extract the plurality of candidates; The medical information processing device according to claim 4 .

6. The processor: extracting the plurality of candidates by inputting text data of the text included in the integrated text region into a trained model that outputs candidates of medical information when text data is input; The medical information processing device according to claim 1 .

7. The processor: determining whether the text included in the text area is medical information based on text data of the text included in the text area; superimposing the text region on the image data based on the likelihood of the text region being medical information; The medical information processing device according to claim 1 .

8. a camera that captures an image of the text indicating the medical information and converts it into image data; a display on which the text area is superimposed on the image data; an input device for accepting a selection of the first text area; Equipped with The medical information processing device according to claim 1 .

9. at least one processor; at least one memory storing instructions for execution by said processor; Equipped with The processor: acquiring image data including text indicating medical information; Extracting text regions from the image data; converting the text contained in the text area into text data; identifying an item of medical information indicated by the text for each of the text areas based on the text data; Medical information processing equipment.

10. The items include at least one of a drug name, dosage, and time point of administration. The medical information processing device according to claim 9 .

11. a database in which master data of medical information for each item is stored; The processor: searching the database using the converted text data to identify the item; The medical information processing device according to claim 9 .

12. The processor: Identifying the items by inputting the converted text data into a trained model that outputs items of medical information when text data is input. The medical information processing device according to claim 9 .

13. At least one processor acquiring image data including text indicating medical information; Extracting text regions from the image data; converting the text contained in the text area into text data; The text area is displayed superimposed on the image data; accepting a selection of a first text area from the plurality of superimposed text areas; Integrating the selected first text area and a second text area that is composed of one or more of the text areas different from the first text area into one text area; displaying a plurality of selectable candidates of medical information indicated by the text in the integrated text area based on text data of the text included in the integrated text area; Medical information processing methods.

14. At least one processor acquiring image data including text indicating medical information; Extracting text regions from the image data; converting the text contained in the text area into text data; identifying an item of medical information indicated by the text for each of the text areas based on the text data; Medical information processing methods.

15. A program that causes a computer to execute the medical information processing method according to claim 13 or 14.

Images