Drug identification support system, drug identification support method, and program

Abstract

Accurately identify chemicals from images containing the word "chemical". [Solution] The drug identification support system comprises an image acquisition unit that captures an image obtained by photographing a string of characters indicating a drug prescribed to an individual, and a drug name output unit that outputs the drug name of at least one drug string that matches a drug name in a drug master from among a group of multiple character strings recognized in the image, as a drug candidate identified in the image. The drug name output unit does not output the drug name of the drug string as a drug candidate if the drug name of the drug string is a pre-set non-output drug.

Description

Technical Field

[0001] The present invention relates to a technology for assisting in the identification of drugs using a computer system.

Background Art

[0002] Japanese Patent Application Laid-Open No. 2024-095140 (Patent Document 1) discloses an image processing system used in a dispensing pharmacy that prepares drugs based on a prescription issued by a medical institution. The image processing system includes a scanner, a server, and an operation terminal. The scanner reads a prescription and generates image data indicating an image G1 of the prescription. The server includes an acquisition unit that acquires an image, an extraction unit that extracts characters from the image, a storage unit that stores a target character string that is a comparison target for the extracted characters, a conversion unit, a calculation unit, and an output unit. The conversion unit converts the extracted character string into a first conversion character string or converts the target character string into a second conversion character string. The calculation unit calculates a first similarity between the first conversion character string and the target character string or calculates a second similarity between the second conversion character string and the extracted character string. The output unit outputs the target character string when the first similarity is equal to or greater than a threshold α or outputs the target character string when the second similarity is equal to or greater than a threshold β.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When an individual receives medical services, healthcare providers may need to keep track of the medications prescribed to that individual. If an individual has been prescribed medications from one or more healthcare institutions, healthcare providers identify the prescribed medications based on the individual's prescription, medication record book, or the information on the medication packaging, or the appearance of the medication itself. This process of identifying medications is called drug identification. Drug identification is performed by qualified professionals such as pharmacists. In recent years, the scope of medical services has expanded, and the locations and situations in which an individual's medications are identified have become more diverse, including healthcare institutions and the individual's home. Drug identification can be supported by using computers to automatically identify medications from images containing text related to medications, such as prescriptions or medication record books.

[0005] Therefore, this application discloses a drug identification support system, a drug identification support method, and a program that can accurately identify drugs from images containing the text of drugs. [Means for solving the problem]

[0006] The drug identification support system in an embodiment of the present invention includes an image acquisition unit that acquires an image obtained by photographing a string of characters indicating a drug prescribed to an individual, and a drug name output unit that outputs the drug name of at least one drug string that matches a drug name in a drug master from among a plurality of recognized string groups in the image as a drug candidate identified in the image. The drug name output unit does not output the drug name of the drug string as a drug candidate if the drug name of the drug string is a pre-set non-output drug. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows an example configuration of the drug identification support system 10 of this embodiment. [Figure 2] Figure 1 is a flowchart illustrating an example of the operation of the drug identification support system 10. [Figure 3] This figure shows examples of the shooting screen and the display screen for the identification results. [Figure 4] This flowchart shows an example of drug identification processing in an image. [Figure 5] This flowchart shows an example of tablet quantity counting processing in a drug identification support system. [Figure 6] This is a perspective view showing an example of the configuration of the drug placement stand in this embodiment. [Figure 7] This figure shows the folded state of the drug placement stand shown in Figure 6. [Modes for carrying out the invention]

[0008] (Composition 1) The drug identification support system in an embodiment of the present invention includes an image acquisition unit that acquires an image obtained by photographing a string of characters indicating a drug prescribed to an individual, and a drug name output unit that outputs the drug name of at least one drug string that matches a drug name in a drug master from among a plurality of recognized string groups in the image as a drug candidate identified in the image. The drug name output unit does not output the drug name of the drug string as a drug candidate if the drug name of the drug string is a pre-set non-output drug.

[0009] The types of drugs included in an image vary depending on the circumstances under which the image containing drug text is acquired. For example, in some cases, personally prescribed medications, such as medications brought by an individual, may be identified from images of documents such as prescriptions or medication records. However, the drug master may include drugs that cannot be prescribed. In such cases, in configuration 1 described above, drugs that cannot be prescribed are set as non-output drugs, preventing unnecessary drug names from being output as drug candidates. This improves the accuracy of drug identification. In other words, it becomes possible to accurately identify drugs from images containing drug text.

[0010] A string that matches a drug name in the drug master is a string with a high similarity to the drug name in the drug master, and is determined to be a match for the drug name in the drug master. For example, the similarity of a string to a drug name in the drug master is calculated, and if the calculated similarity satisfies the conditions, the string is determined to be a match for the drug name in the drug master. The drug master contains multiple drug names. For example, for each of the multiple string groups, the similarity is calculated for each of the multiple drug names. For strings among the multiple string groups that are determined to be a match for a drug name in the drug master, the drug name that matches that string is output as a drug candidate.

[0011] When the drug name in the aforementioned drug string is a pre-set non-output drug, the form in which the drug name in the aforementioned drug string is not output as a drug candidate includes not only the form in which the drug name is not output at all, but also the form in which it is output separately from other drug candidates that are considered to have low similarity.

[0012] (Configuration 2) In the above configuration 1, the drug identification support system may further include a non-output drug setting unit that sets the non-output drugs based on input from the user. This allows the user to set appropriate non-output drugs according to the situation.

[0013] (Composition 3) The drug identification support system in an embodiment of the present invention comprises an image acquisition unit that acquires an image containing a string of characters indicating a drug prescribed to an individual, and a drug name output unit that outputs the drug name of at least one drug string that matches a drug name in a drug master from among a plurality of recognized string groups in the image as a drug candidate included in the image. The drug name output unit outputs the drug name of a drug string that matches a drug name in a drug master from among the string groups obtained by excluding from the plurality of string groups lines containing a predetermined number of characters including a pre-set exclusion designation character at the beginning, as a drug candidate included in the image.

[0014] For example, in documents such as medication records, drug names are sometimes listed together with their generic names. Often, the generic name is preceded by a notation such as "(generic)". There are various ways to list drug names; for example, "(alternative)" to indicate a substitute drug or "(generic)" to indicate a generic drug may be added to the drug name. Also, in identifying drugs in documents, for example, the generic name of a drug may not be necessary. In this case, in configuration 3 above, for example, "generic" can be set as an excluded character. Then, for example, the drug name output unit can be configured to output drug names as drug candidates from the group of strings that match drug names in the drug master, excluding strings in lines where the first three characters contain "generic". This prevents the generic name of a drug from being output as a drug candidate. In this way, configuration 3 above can reduce the over-detection of candidate drugs and improve the accuracy of drug identification.

[0015] (Composition 4) In the above configuration 3, the drug identification support system may further include an exclusion character setting unit that sets the exclusion character based on input from the user. This allows the user to set an appropriate exclusion character according to the situation.

[0016] The drug identification support method in an embodiment of the present invention includes the following steps performed by a computer. The drug identification support method includes an image acquisition step of acquiring an image obtained by photographing a string of characters indicating a drug prescribed to an individual, and a drug name output step of outputting the drug name of at least one drug string that matches a drug name in a drug master from among a plurality of string groups recognized in the image as a drug candidate identified in the image. In the drug name output step, if the drug name of the drug string is a pre-set non-output drug, the drug name of the drug string is not output as a drug candidate.

[0017] The program in the embodiment of the present invention causes a computer to execute an image acquisition process for acquiring an image obtained by photographing a string indicating a drug prescribed to an individual, and a drug name output process for outputting, as a drug candidate, the drug name of at least one drug string that matches the drug name in the drug master among a plurality of string groups recognized in the image. In the drug name output unit process, when the drug name of the drug string is a preset non-output drug, the drug name of the drug string is not output as a drug candidate.

[0018] The image recording unit may receive, from the user, a designation of the form of the drug in the image together with the image of the drug. The image recording unit may record, in the storage unit in association with the image, the form of the drug designated by the user. The identification result acquisition unit may select an identification process for the drug in the image in the identification system according to the form of the drug in the image. For example, the identification result acquisition unit may provide, to the identification system, information indicating the form of the drug in the image together with the image of the drug. The identification result acquisition unit can obtain an identification result of an identification process according to the form of the drug in the image.

[0019] The designation of the form of the drug may, for example, be a designation of whether the object of photographing is the drug itself or text indicating information about the drug. When the object of photographing is the drug itself, the form may be further specified in detail according to the state of the drug. The forms of the drug for which designations are received may, as an example, include at least one of the drug itself, text indicating information about the drug, and the code of the drug. For the drug itself, the state of the drug itself, such as a naked tablet, a drug package, a PTP sheet, etc., may also be specified. For the text of the drug, the type of the document on which it is written, such as a prescription, a medicine bag, or a medicine notebook, etc., may also be specified. For the code, the type of the code, such as GS1, QR code (registered trademark), etc., may also be specified. Thereby, an appropriate identification process can be executed according to the form of the drug shown in the image. Also, the identification work for drugs in more diverse states becomes easier. Note that a naked tablet is a tablet or capsule in an exposed state not packaged in a drug package or the like.

[0020] The image recording unit may accept a drug photographing operation by the user. That is, the image recording unit may record an image obtained by a drug photographing operation by the user in the storage unit. The image recording unit can accept a specification of the form of the drug from the user together with the photographing operation by the user. In this case, the photographing mode can be switched according to the specification of the form of the drug. By switching the photographing mode, for example, the screen for accepting the photographing operation may be switched.

[0021] (Configuration 7) An identification system that executes drug identification processing based on a drug image is also included in an embodiment of the present invention. The identification system acquires a photographed image of a drug placed on a placement surface provided with a plurality of markers including combinations of two or more reference colors, and performs trapezoidal correction on the photographed image based on the positions of the images of the plurality of markers, and color correction based on the colors in the images of the combinations of the reference colors of the plurality of markers, respectively, to generate a corrected image, a drug region extraction unit that extracts a drug region from the corrected image, and an identification unit that identifies the drug in the drug region based on a partial image of the drug region. In this identification system, trapezoidal correction and color correction can be performed using the markers included in the photographed image of the drug. As a result, the influence on the identification accuracy due to the drug photographing conditions is suppressed. As a result, it becomes easy to identify drugs in various places or situations. The drug identification support system of the above Configurations 1 to 5 may further include the identification system.

[0022] Hereinafter, embodiments of the present invention will be described while referring to the drawings. [Embodiment] (Example of System Configuration) Figure 1 shows an example configuration of the drug identification support system 10 of this embodiment. The drug identification support system 10 acquires an image of a string of characters indicating a drug prescribed to an individual and outputs drug candidates recognized in the image. The drug identification support system 10 is data-communicable with the identification system 20. The identification system 20 identifies the drugs included in the image. In the example in Figure 1, the drug identification support system 10 provides an image of the drugs to the identification system 20 and obtains the drug identification result from the identification system 20. The drug identification support system 10 and the identification system 20 can access the storage unit 6. The storage unit 6 stores data of drug masters, non-output drugs, and excluded designated characters. The drug identification support system 10 is wirelessly or wiredly connected to the input / output device 7 and the camera 8. The input / output device 7 includes a device for the user to input information and a device for outputting information to the user.

[0023] (Example configuration of a drug identification support system) The drug identification support system 10 includes a shooting interface unit (hereinafter referred to as the shooting IF unit) 1, an image acquisition unit 2, a drug name output unit 3, a non-output drug setting unit 4, and an exclusion designation character setting unit 5.

[0024] The imaging interface unit 1 provides an interface for the user to perform imaging operations on pharmaceuticals. For example, the imaging interface unit 1 displays the monitor of the camera 8 on the display of the input / output device 7. The imaging interface unit 1 also receives imaging operations from the user via the input / output device 7. The imaging interface unit 1 may also accept a specification of the form of pharmaceuticals to be photographed in conjunction with the user's imaging operation. The form of pharmaceuticals that can be specified may include, as an example, at least one of the following: the pharmaceutical itself, a string of characters indicating the pharmaceutical, or a pharmaceutical code. In the example in Figure 1, the form of pharmaceuticals to be photographed can be any of the following: pharmaceutical D as an unpackaged tablet placed on the pharmaceutical stand 50 or pharmaceutical D contained in a pharmaceutical package H, a barcode (GS1 code) or 2D code indicating pharmaceutical information, or a string of characters indicating the pharmaceutical. The string of characters indicating the pharmaceutical is a string of characters indicating pharmaceuticals prescribed to an individual. The string of characters indicating the pharmaceutical is written, for example, on a prescription, a medicine bag, or a medication record book. The type of document on which these pharmaceutical strings are written may also be specified.

[0025] The image acquisition unit 2 acquires an image of the drug to be identified. The acquired image may be, for example, the drug itself, a string of characters indicating the drug, or the drug's code. The image is stored in, for example, the storage unit 6. The image acquisition unit 2 may also acquire an image by, for example, receiving input of an image of the drug taken by the camera 8 through user operation and storing it in the storage unit 6.

[0026] The image acquired by the image acquisition unit 2 is provided to the identification system 20, which performs the identification process. The drug identification support system 10 obtains the drug identification results for the drugs contained in the image from the identification system 20. The identification results include information on drug candidates identified in the image. For example, if the form of the drug being photographed is a string of characters indicating the drug, the drug name that matches the string recognized in the image is included in the identification results as a drug candidate. For example, among a group of strings recognized in the image, for drug strings that are determined to match a drug name in the drug master, the drug name that matches this drug string (i.e., the drug name in the drug string) is included in the identification results as a drug candidate. Note that for a single drug string in the image, multiple candidate drugs may be included in the identification results.

[0027] The drug name output unit 3 outputs the identification result. The drug name output unit 3 displays the candidate drugs recognized in the image on the screen of the input / output device 7. For example, if the image is an image obtained by photographing the string of drug names, the candidate drugs that match the string recognized in the image will be displayed. Note that the output format of candidate drugs by the drug name output unit 3 is not limited to display. For example, candidate drugs may be output in the form of printing, data transmission, data storage, audio output, or other forms.

[0028] The drug name output unit 3 is configured not to output the drug name of a drug string recognized in an image as a drug candidate if that drug name is a pre-set non-output drug. For example, non-output drugs are set when they are stored in the storage unit 6. In this case, the identification system 20 is configured not to consider non-output drugs in the storage unit 6 as drug candidates to be identified in the image. This allows the drug name output unit 3 to avoid outputting non-output drugs as drug candidates. A specific example of this will be described later.

[0029] The non-output chemical setting unit 4 receives input of non-output chemicals from the user and sets the non-output chemicals by storing the non-output chemicals entered by the user in the storage unit 6. The non-output chemical setting unit 4 can receive input of non-output chemicals from the user, for example, via the input / output device 7.

[0030] The drug name output unit 3 outputs drug names that match drug names in the drug master from the group of strings obtained by removing from the group of strings recognized in the image lines that contain a predetermined number of characters including a pre-set exclusion character at the beginning, as drug candidates included in the image. As a result, strings with a predetermined number of characters including a specified character at the beginning are excluded from drug identification. For example, an exclusion character is set when an exclusion character is stored in the storage unit 6. The identification system 20 can perform drug identification processing on the group of strings obtained by removing from the group of strings recognized in the image lines that contain a predetermined number of characters including an exclusion character at the beginning, and determine drug candidates. As a result, the drug name output unit 3 can output drug candidates in the group of strings obtained by removing the part indicated by the exclusion character from the recognized group of strings.

[0031] The exclusion character setting unit 5 accepts exclusion character input from the user and sets the exclusion character by storing the user-entered exclusion character in the storage unit 6. The exclusion character setting unit 5 can accept exclusion character input from the user, for example, via the input / output device 7.

[0032] (Example of identification system configuration) In the example shown in Figure 1, the identification system 20 includes, as an example, a character recognition unit 21 and a drug determination unit 22. The character recognition unit 21 recognizes characters in the image. The drug determination unit 22 compares the character string recognized by the character recognition unit 21 with the drug names in the drug master and determines drugs with a high degree of similarity (i.e., match) to the string as candidate drugs. By determining candidate drugs, the drugs in the image are identified by the drug determination unit 22. As an identification result, the drug determination unit 22 outputs information about candidate drugs, i.e., candidate drugs, that are identified in the image. The candidate drug information includes, for example, the drug name. For example, for each drug identified in the image, information about multiple candidate drugs that satisfy the condition of match with the drug master may be output as an identification result.

[0033] In the example shown in Figure 1, the character recognition unit 21 and the drug determination unit 22 of the identification system 20 identify a drug in an image containing a string of characters indicating the drug. The identification system 20 may also be configured to identify a drug in an image obtained by photographing the drug itself. In this case, for example, the identification system 20 has a correction unit, a drug area extraction unit, and an identification unit. The correction unit acquires a photographed image of a drug placed on a mounting surface provided with multiple markers, and generates a corrected image by applying trapezoidal correction based on the positions of the multiple markers in the images and color correction based on the colors of the multiple markers in the images to the photographed image. The drug area extraction unit extracts the drug area from the corrected image. The identification unit identifies the drug in the drug area based on a partial image of the drug area. The identification system 20 may also be configured to identify a drug in an image obtained by photographing the drug's code.

[0034] The identification system 20 receives an image along with information indicating the form of the drug in that image, and can perform identification processing according to the form of the drug. For example, if the form of the drug in the target image is the drug itself, the identification system 20 performs identification processing on the image of the drug itself, and if the form of the drug is a string of characters indicating the drug, it performs identification processing on the image of the string of characters indicating the drug. If the form of the drug is specified in more detail, such as an unwrapped tablet or a drug packet, identification processing may be performed according to each of the detailed forms of the drug, such as an unwrapped tablet or a drug packet.

[0035] The drug identification support system 10 and the identification system 20 can be configured by one or more computers. Each functional unit of the drug identification support system 10 and the identification system 20 can be realized by the computer's processor executing a program in memory. Such a program and a non-transitory storage medium storing the program are also included in embodiments of the present invention. The storage unit 6 can be configured by one or more storage and / or memory accessible by the computers constituting the drug identification support system 10.

[0036] For example, the drug identification support system 10 is implemented in a mobile terminal such as a smartphone. The identification system 20 is implemented in a computer capable of communicating with the mobile terminal. The mobile terminal 30 has, for example, a processor, memory, storage device, touch panel (an example of an input / output device), and camera. The computer on which the identification system 20 is implemented has, for example, a processor, memory, and storage device. In this case, the storage unit 6 in Figure 1 consists of the storage device of the mobile terminal and the storage device of the computer. Thus, the storage unit 6 may consist of a storage device accessible from both the drug identification support system 10 and the identification system 20. The drug identification support system 10 may also include the identification system 20. For example, the drug identification support system 10 may consist of a computer such as a server or edge computer capable of communicating with a mobile terminal having a camera and input / output devices.

[0037] (Example of operation) Figure 2 is a flowchart showing an example of the operation of the drug identification support system 10 shown in Figure 1. In the example in Figure 2, at S1, the drug identification support system 10 displays an operation menu on the display of the input / output device 7 (S1). If the user selects a setting in the operation menu (YES at S2), a reception screen G1 is displayed that accepts input for non-output drugs and excluded characters (S3). In the example in Figure 2, the input for non-output drugs and excluded characters is accepted on a single screen. As a variation, the input for non-output drugs and excluded characters can be accepted on separate, independent screens. The non-output drugs and excluded characters entered by the user at S2 are stored in the storage unit 6. This sets the non-output drugs and excluded characters (S4).

[0038] If the user selects "Capture" on the menu screen (YES in S4), the image acquisition unit 2 accepts image input from the user (S6). In this example, in S6, the capture interface unit 1 displays a capture screen for the user to photograph the drug. The capture interface unit 1 allows the user to input the type of drug to be photographed. The capture interface unit 1 switches the capture screen according to the type of drug specified by the user.

[0039] Figure 3(a) shows an example of the capture screen. Figure 3(a) shows the capture screen when a string of characters (OCR) indicating the drug is specified as the form of the drug to be captured. In the example of Figure 3(a), in addition to the string of characters (OCR), unwrapped tablets, drug packets, GS1 codes (GS1), and QR codes (registered trademarks) (QR) can also be specified as the form of the drug to be captured. In any case, the capture screen includes a monitor image A1 that displays the image within the camera 8's shooting range in real time. Below, an example of identifying a drug in an image containing a string of characters indicating the drug will be explained.

[0040] In the example shown in Figure 3(a), the shooting IF unit 1 displays a targeting position mark C1 on the monitor image A1, which displays the shooting range of the camera 8, indicating the targeting position specified by the user's specified operation. For example, the position where the user long-tap on monitor image A1 becomes the targeting position. The user specifies the position of the drug's text string on monitor image A1 at the time of shooting as the targeting position. Alternatively, the user takes a picture with the drug's text string at the position of the targeting position mark C1 on monitor image A1. Data indicating the targeting position in the captured image is supplied to the identification system 20 along with the image. The identification system 20 uses the data indicating the targeting position to identify the drug indicated by the text string in the image. For example, the drug recognition process in the image is performed using the coordinates of the targeting position in the image. That is, the targeting position in the image specified by the user is used in the drug recognition process in that image. By linking the targeting position to the image processing for drug recognition, drugs can be identified more efficiently.

[0041] For example, the drug determination unit 22 of the identification system 20 can perform character recognition and matching with a drug master for areas of the image where the position relative to the aiming position satisfies predetermined conditions. In other words, for areas where the position relative to the aiming position does not satisfy predetermined conditions, the process for identifying the drug can be omitted. For example, after detecting areas of characters from a captured image, the system can avoid reading characters whose starting position in the row direction (i.e., horizontal direction) does not match the character at the aiming position. This allows the system to ignore characters written in columns other than the column containing the drug name. Therefore, the efficiency and accuracy of the drug recognition process can be further improved. In this example, the aiming point is not fixed to the center of the monitor image, but can be specified by the user at any position in the image. Therefore, the user can specify the position of the drug name as the aiming point, regardless of the format of the document containing the drug name (e.g., medication record book). Note that the operation to specify the aiming point is not limited to a long tap. For example, the user may instruct the autofocus to focus on the drug name. In this case, the point of focus can be used as the aiming point.

[0042] The shooting IF unit 1 can enlarge the monitor image A1, which displays the image of the shooting range of the camera 8 as shown in Figure 3(a), in response to user operation. For example, the operation to instruct the enlargement of the monitor image may be, for example, pressing the volume up button on the mobile terminal that constitutes the drug identification support system 10. The enlargement may be performed, for example, based on the center of the monitor image A1. Alternatively, the monitor image A1 may be enlarged based on the aiming position specified by the user. The enlargement makes it easier for the user to confirm whether the drug text is in focus.

[0043] In S6 of Figure 2, the image acquisition unit 2 acquires the image captured by the user's shooting operation. The image acquired by the image acquisition unit 2 is provided to the identification system 20 (S7). The identification system 20 performs drug identification processing on the provided image (S8). The drug name output unit 3 obtains the identification result from the identification system 20 (S9). The identification result includes information indicating candidate drugs included in the image. The drug name output unit 3 outputs the identification result (S10). For example, the drug name output unit 3 displays the candidate drugs indicated by the identification result along with the image on the display of the input / output device 7.

[0044] The drug name output unit 3 may display the recognized text area in the image and accept the user's selection of the text area containing the drug name. In the example shown in Figure 3(b), image A2 is displayed, in which rectangles indicating the recognized text area in the image are superimposed on the captured image. In image A2, the rectangles indicating the text area are displayed in a manner that allows the user to select. The user can select rectangle D1 in image A2 that indicates the text area containing the drug name.

[0045] Data indicating a string area containing the string of the drug selected by the user is supplied to the identification system 20. The identification system 20 uses the data indicating the string area containing the drug string to identify the drug indicated by the string in the image. This improves processing efficiency. The identification system 20 can perform drug recognition processing that takes into account the string area selected by the user. For example, the range of strings to be processed for drug identification can be limited to an area based on the width of the selected string area. As an example, processing can be limited to strings in the area extending above and below the selected string area (horizontal width). This allows, for example, strings that have been combined with extraneous characters to be separated and processed. Alternatively, the drug determination unit 22 can relax the similarity conditions in matching the string with the drug name in the area based on the selected string area and perform the matching process again.

[0046] Figure 3(c) is an example of a screen displaying candidate drugs shown by the drug name output unit 3. In the example in Figure 3(c), image A3 is displayed, in which a rectangle D2 indicating the region recognized as containing drug strings is superimposed on the captured image. In addition, drug names D3 that match the drug strings in each rectangle D2 are displayed. In image A3, either the rectangle D2 indicating the region of the drug string or the drug name D3 that matches the drug string is displayed in a manner that allows the user to select either the drug string D2 or the drug name D3. When the user selects either the drug string D2 or the drug name D3, detailed drug candidate information for the selected drug string D2 is displayed.

[0047] Figures 3(d) and 3(c) are examples of screens that display drug candidates matching the drug string selected by the user. In Figures 3(d) and 3(c), for one drug included in image A3, the image of that drug and information on one or more candidate drugs are displayed in a selectable format. The user can input confirmation of the identification result by selecting one of the candidate drugs. In the example of Figure 3(d), the user selects one of the candidate drugs and presses button B2 to confirm, thereby inputting confirmation of the identification result. As a result, the identified drug is recorded as confirmed, i.e., identified and confirmed. In other words, the user's confirmation result is recorded (S11 in Figure 2). Thus, the drug name output unit 3 may display candidate drugs to the user along with their images and accept input from the user for confirmation of the candidate drugs.

[0048] Figure 3(c) shows an example of a case where a drug name is incorrectly detected. In this example, since "cholesterol" is included in the drug master as a drug name, "cholesterol" in the drug description is incorrectly detected as a drug candidate. Since "cholesterol" is not a drug that is expected to be prescribed, it is preferable to set it as a non-output drug. In the example of Figure 3(c), a tool (button B3 as an example) for setting the displayed drug candidate as a non-output drug is provided on the screen that displays drug candidates. By pressing button B3, the user can input an instruction to add the displayed drug candidate ("cholesterol") to the non-output drugs. In this way, the non-output drug setting unit 4 may receive an instruction from the user to set the drug candidate output by the drug name output unit 3 as a non-output drug. When an instruction to set the outputted drug candidate as a non-output drug is input by the user (YES in S12 of Figure 2), the non-output drug setting unit 4 stores that drug candidate as a non-output drug in the storage unit 6 (S13).

[0049] Figure 4 is a flowchart illustrating an example of the drug identification process in an image by the identification system 20. In the example in Figure 4, the character recognition unit 21 detects character regions in the image (S801). Character region detection can be performed using, for example, connected component analysis, sliding window, MSER (Maximally Stable Extremal Regions), or deep learning-based methods, although these methods are not particularly limited. Examples of deep learning-based methods include models such as EAST (Efficient and Accurate Scene Text) or CRAFT (Character Region Awareness for Text Detection). As a result of detecting character regions, for example, rectangular data surrounding each character region is generated.

[0050] The character recognition unit 21 recognizes the characters in the detected character string region. While not particularly limited, the character recognition process can utilize pattern recognition such as template matching, HMM models, or deep learning-based methods such as CNN, RNN, or LSTM. Character recognition determines the character string in each character region. This generates multiple sets of strings. Hereinafter, each of these multiple string sets, as a processing unit, may be referred to as a "string box."

[0051] The character recognition processing in S802 may be limited to the area based on the user-specified target in Figure 3(a), among all the character string areas detected in S801. For example, the character recognition processing may be limited to the character string area in the area extending vertically from the character string area at the target position, with a width equal to that character string area. Alternatively, the character recognition processing may be limited to the area based on the character string area selected by the user in Figure 3(b).

[0052] The drug determination unit 22 performs a matching process with the drug name in the drug master for each of the multiple string groups. The matching process includes, for example, a process to calculate the similarity (i.e., degree of match) between the target string and the drug name in the drug master. A score is calculated as a value indicating the similarity. A drug name in the drug master that satisfies the condition in terms of similarity to the string is determined to be a drug name that matches the string. The matching process with the drug master may be limited to string groups within a range based on at least one of the targeting position specified by the user (see Figure 3(a)) and the string area selected by the user (see Figure 3(b)).

[0053] In the example shown in Figure 3, the drug determination unit 22 repeatedly executes the loop processing from S803 to S814 for each target string box. In S803, it is determined whether the target string contains a predetermined number of characters that include a specified exclusion character at the beginning. If the drug determination unit 22 finds a predetermined number of characters that include a specified exclusion character at the beginning (YES in S803), it excludes the strings on the same line as that string from the matching process (S814).

[0054] For example, medication notebooks often list the generic name of a drug along with its product name, often preceded by a notation such as "(generic)". The generic name may also match the drug name in the drug master. In this case, it results in false positives. Therefore, by pre-setting "generic" or similar characters as excluded characters, and excluding lines containing these excluded characters within a predetermined string of characters at the beginning of the entry, false positives of generic names can be reduced. Since the notation of generic names varies from one medication notebook to another, the excluded character setting unit 5 allows users to add, delete, and modify excluded characters.

[0055] In S804, the drug determination unit 22 performs a matching process for the target string. For example, the similarity between the target string and all drug names in the drug master is calculated, and based on the similarity, it is determined whether or not the target string matches a drug name in the drug master. The similarity is not particularly limited, but for example, it can be calculated using the Levenshtein distance between the string and the drug name in the drug master.

[0056] The processes S805 to S810 in Figure 3 are for calculating the similarity of strings formed by concatenating the target string with adjacent strings as needed. In S805, it is determined whether or not to search for adjacent strings, that is, whether or not to concatenate the strings. This determination is made based on the partial similarity between the target string and all drug names in the drug master. For example, if the drug names in the drug master are longer than the target string, and the partial similarity of both is higher than a threshold, it is determined to search for adjacent string boxes (YES in S805). Partial similarity is calculated, for example, if the string lengths of the strings to be compared are different, by shifting the shorter string one character at a time relative to the longer string and comparing it with the portion of the longer string that has the same string length as the shorter string, and using the similarity at the position where the similarity is highest.

[0057] A search for adjacent string boxes is performed (S806), and if an adjacent string box is found as a result of the search (YES in S807), the target string and the adjacent string are combined (S808). The drug determination unit 22 calculates the similarity between the combined string and the drug name in the drug master (S809). Based on the calculated similarity, it is determined whether or not to search for further adjacent strings (S810). The decision in S810 is made based on partial similarity, similar to S805, as well as whether or not the number of searches has reached the upper limit. Note that the search process may also be performed, for example, based on whether the positional relationship between the rectangular area of ​​the target string box and the rectangular areas of string boxes within a predetermined distance to the right and downward satisfies predetermined conditions.

[0058] The processing in S805-S810 determines whether the target string and the drug name in the drug master have a high partial similarity. In this case, the string is combined with the string to the right or below (on the next line), and the similarity is calculated for the combined string. This allows for accurate identification of drug names, even if they are written with line breaks. In other words, in this example, the drug determination unit 22 calculates the partial similarity of both strings when calculating the similarity between each string and the drug name in the drug master. If the partial similarity is high, the string on the next line is combined, and the similarity of the combined string is calculated. This allows for accurate identification of drug names even with line breaks. Furthermore, drug names can be accurately identified even when they are contained within text. Note that if the length of the combined string is longer than the length of the drug name in the drug master, it is preferable to use the partial similarity.

[0059] The drug determination unit 22 determines candidate drugs based on the similarity of the target string or combined string (S811). If the similarity between the target string or combined string and a drug name in the drug master satisfies predetermined conditions, that drug name is determined as a candidate drug as a drug name that matches the target string or combined string. When strings are combined, drug names that match the target string and the combined string whose similarity satisfies the conditions are determined as candidate drugs. Multiple drug names may be determined as candidate drugs for a single string.

[0060] The drug determination unit 22 compares the determined candidate drug with the non-output drugs in the memory unit 6. If it determines that a candidate drug is a non-output drug (YES in S812), it removes that candidate drug from the list of candidate drugs (S813). This ensures that non-output drugs are not included in the identification results as candidate drugs.

[0061] Once the processing steps S803 to S813 for each of the string groups to be processed is completed, the drug determination unit 22 outputs the candidate drug names determined for each string group as identification results (S815). For example, the identification results are stored in the storage unit 6, which is accessible to the drug identification support system 10.

[0062] The matching process between the target string and the drug name in the drug master is not limited to the example above. For example, a process may be further performed to convert frequently occurring characters in at least one of the target string and the drug name in the drug master to shorter characters (e.g., one character). In this case, the similarity between the converted target string and the drug name in the drug master is calculated. For example, "capsule," "hydrochloride tablet," and "250mg" are common strings for many drugs. In strings containing such common strings with weak distinctiveness, the match of the common string with the drug master will increase the similarity score. For example, if the target string is "Tafmac Capsule," if the "Tafmac" part is slightly missing, the similarity to other "xxxxxx capsules" will be calculated as higher than that of "Tafmac Capsule." Therefore, for example, in the drug name in the drug master, the common string can be replaced with a single character that is not normally used, such as "capsule" → "%." This increases the degree to which the match of the distinctive "Tafmac" part with the drug name in the drug master contributes to the similarity score. As a result, the accuracy of drug identification improves. For example, drug names similar to the target string are more likely to appear higher in the list of candidate drugs.

[0063] Many drug names end with "tablet," "capsule," "~mg," or the manufacturer's name. Therefore, the latter part of a drug name has less distinctiveness than the former part. In order to calculate the similarity between a target string and a drug name in the drug master, the similarity may be calculated by weighting the first part of the drug name more than the latter part, so that the degree of matching of the latter part of the drug name contributes more to the similarity score. For example, when calculating the Levenshtein distance, the penalty for swapping the first part of a string can be made heavier than that for swapping the latter part, i.e., weighted so that the score drops more, while the penalty for swapping characters in the latter part can be made lighter than that for swapping the first part, i.e., weighted so that the score does not drop as much as with swapping the first part. The first part may be, for example, the first number of characters from the beginning of the string, and the latter part may be the last number of characters from the end of the string. Alternatively, the first part of the string may be the first part and the second part may be the latter part.

[0064] In the image, strings of a specific format that have been set in advance can be excluded from the matching process. This improves matching accuracy. The specific format can be, for example, a string consisting of a number followed by "mg". Drug names listed in medication notebooks are often in the format "XXX tablets △△mg pharmaceutical company name" or "XXX capsules △△mg pharmaceutical company name". In the case of a drug name in the format of ingredient name + space + ingredient amount mg, the space in between may cause the ingredient name and ingredient amount mg ​​to be recognized as separate strings. In this case, if the matching process is performed on the string of ingredient amount mg, drugs with completely different ingredient names but the same ingredient amount are more likely to be selected as candidate drugs. By excluding strings consisting only of a number + "mg" from the matching process as a specific format, the probability of completely different drug names being selected as drug candidates is reduced. In this way, by excluding the specific format in the drug determination unit 22, the drug name output unit 3 can output drug names that match drug names in the drug master from the group of strings obtained by excluding the pre-set specific format from the group of recognized strings as drug candidates. The drug identification support system 10 or the identification system 70 may further include a specific format setting unit that sets the specific format based on input from the user.

[0065] The drug identification support system may have a tablet counting function. Specifically, the drug identification support system may be configured to include an image acquisition unit and a tablet count output unit. The image acquisition unit acquires images obtained by photographing tablets. In addition to the image of the tablets, the image acquisition unit may also acquire images of text indicating the tablets, or one-dimensional or two-dimensional codes indicating the tablets. The tablet count output unit outputs the number of tablets recognized in the images acquired by the image acquisition unit. The tablet count output unit may display a recognition result image on a display, in which marks indicating the positions of the tablets recognized in the image are superimposed on the images of the photographed tablets. The recognition result image is displayed, for example, for visual confirmation by the user.

[0066] The tablet count output unit can change the display format of the marks in the recognition result image in response to user operation. This makes it easier for the user to visually confirm the count. The display format of the marks that can be changed in response to user operation may be, for example, at least one of the mark's position relative to the tablet, the mark's color, and the mark's transparency. The user operation that triggers the switching of the mark's display format is preferably a simple operation, such as tapping or clicking a predetermined area on the screen.

[0067] The tablet count output unit can change the display format of the marks in the recognition result image according to the zoom level of the recognition result image. The display format of the marks to be changed according to the zoom level may be, for example, at least one of the position of the marks relative to the tablets and the degree of transparency of the marks. The zoom level that triggers the change in the display format of the marks may be, for example, when the degree of magnification exceeds a predetermined value.

[0068] The tablet count output unit can output information indicating the quantity of tablets in a format readable by the packaging machine. For example, the tablet count output unit may display a one-dimensional or two-dimensional code containing information identifying the recognized tablet and information indicating the quantity of that tablet on a display or print it on a printer. The packaging machine reads the tablets and the quantity of tablets output by the tablet count output unit. Based on the read information, the packaging machine can automatically identify the cassette to be filled with the tablets, verify the tablets to be filled, and record the quantity of tablets to be filled. This enables the packaging machine to manage the number of tablets to be filled into the cassette.

[0069] The image acquisition unit may accept input from the user specifying whether to count a single drug or multiple drugs for the acquired image. In this case, the tablet count output unit may change the display of the marks according to whether it is a single drug count or multiple drug count when displaying the recognition result image. For example, if a single drug count is specified, the marks can be displayed in a way that triggers an alert if the recognized tablets contain multiple types of drugs. If multiple drug counts are specified, the marks on all recognized tablets can be displayed in the same way, or in a way that is independent of the type of drug. This makes it possible to display appropriate recognition results according to both situations: counting a single drug (i.e., one type of drug) and counting when multiple drugs are mixed together. For example, when counting a single drug, the risk of contamination by multiple drugs can be reduced.

[0070] Figure 5 is a flowchart illustrating an example of the tablet quantity counting process in the drug identification support system 10. In the example in Figure 5, the image acquisition unit 2 acquires an image of the tablet (S101). For example, an image of the tablet captured by the camera 8 is acquired. The image acquisition unit 2 may also acquire an image of the tablet's GS1 code or a string of characters indicating the tablet written in a medication record book or the like.

[0071] In S102, the drug identification support system 10 receives input from the user specifying whether to perform single-drug counting or multiple-drug counting for the image acquired in S101. For example, a screen is displayed that allows the user to select either single-drug counting mode or multiple-drug counting mode.

[0072] In S103, the drug identification support system 10 acquires information identifying the tablets recognized in the image acquired in S101 and information indicating the quantity of those tablets. The tablet recognition process in the image and the counting process of the recognized tablets may be performed by the identification system 20 or by the drug identification support system 10. The tablet recognition process is not particularly limited, but for example, it may be pattern matching using a master image of the drug master, pattern matching without using a master image, or recognition processing using a learning model generated by machine learning.

[0073] In S104, the tablet count output unit displays a recognition result image on the display of the input / output device 7, which is an image of the captured tablets with marks indicating the position of the tablets superimposed on it, and accepts the user's visual confirmation result. Here, the marks are displayed in a display format according to the single-drug counting or multiple-drug counting mode selected in S102. In single-drug counting mode, if the recognized tablets include tablets with different shapes, the mark display format for the tablets with fewer shapes is made different from the mark display format for the other tablets and displayed as an alert. This reduces the risk of contamination. In multiple-drug counting mode, the marks are displayed in the same display format regardless of the shape of the recognized tablets.

[0074] When the recognition result image is displayed, if the user performs a mark switching instruction operation (YES in S105), the tablet count output unit switches the display format of the marks (S106). The mark switching instruction operation can be a simple one-step operation, such as a single tap or double tap on the screen. Each time a mark switching instruction operation is performed, the tablet count output unit can switch the display format in a predetermined order. For example, as shown in Figures 5(a) to (d), each time the user taps the screen, the mark color can be changed in the order of red superimposed mark, red outline mark, blue superimposed mark, blue outline mark, ... This makes it possible to display various drugs in a variety of ways, such as displaying the marks and drug colors so that they do not overlap, or displaying the outlines clearly. As a result, it becomes easier for the user to reliably visually confirm that there are no counting errors.

[0075] When displaying the recognition result image, if the zoom state of the recognition result image meets a predetermined condition (YES in S107), the tablet count output unit changes the display format of the marks (S108). This allows the display format of the marks to be controlled so that the tablet identification information does not become difficult to see when the user changes the zoom using pinch-in and pinch-out gestures. For example, the display format of the marks can be switched according to the degree of zoom. As an example, as shown in Figures 5(e) to (h), the display format of the marks is changed so that the transparency of the marks increases as the degree of zoom increases. As another variation, when the degree of zoom is below a threshold, the marks can be superimposed on the inside of each tablet, and when the degree of zoom is above the threshold, contour marks can be used to show the outline of the tablets. In this way, by changing the display format of the marks according to the zoom, visibility is improved in both enlarged and normal display states. As a result, it becomes easier for the user to reliably visually confirm that there are no counting errors.

[0076] The user can visually inspect the displayed recognition result image to determine whether all tablets in the image have been recognized and counted. The user inputs the result of the visual inspection by, for example, tapping the OK or NG button on the screen displaying the recognition result image. If the visual inspection result entered by the user is NG (NO in S109), the system accepts input from the user for tablet count correction (S110). Alternatively, if the visual inspection result is NG, the system may return to the image acquisition process in S101 or the tablet count acquisition process in S103 and perform the tablet counting process by image recognition again with different conditions.

[0077] The tablet count output unit outputs the number of tablets that have been visually confirmed as the tablet count result (S111). For example, information identifying the tablet and information indicating the number of tablets is output. The output format of the tablet count is not particularly limited and the tablet count can be output by display, audio output, printing, data storage, data transmission, or other forms.

[0078] For example, when the packaging machine uses information about the number of tablets (YES in S112), the tablet count output unit displays or prints a one-dimensional or two-dimensional code containing information about the recognized tablets and the quantity of those tablets. The information that identifies the tablets included in the code can be information about the tablets recognized based on the image acquired in S101 (e.g., the appearance of the tablets or the GS1 code). For example, if the user instructs the packaging machine to output tablets, S112 is determined to be YES. For example, the user can count the tablets to be filled into the packaging machine's cassette using the drug identification support system 10. In this case, the counting result of the drug identification support system 10 is output as a code. The packaging machine reads the output code and automatically identifies, verifies, and records the quantity to be filled into the cassette. This prevents filling errors.

[0079] Figure 6 is a perspective view showing an example of the configuration of the drug placement stand 50 in this embodiment. The drug placement stand 50 is a stand on which drugs to be photographed are placed. The drug placement stand 50 comprises a base 51, a support portion 52 extending upward from the base 51, a holding portion 53 attached to the support portion 52 on the opposite side of the base 51, a light 54, and a tray 55. The tray 55 has a placement surface on which tablets can be placed. The tray 55 is placed on the base 51. The tray 55 is detachable from the base 51. By removing the tray 55, a sachet can be placed on the base 51. That is, the surface of the base 51 on which the tray 55 is placed is flat. A groove 55a is provided on the placement surface of the tray 55. The holding portion 53 is configured to hold the photographing device 40. The photographing device 40 may be, for example, a mobile terminal such as a smartphone. The support section 52 allows the positional relationship between the tray 55 mounting platform and the imaging device 40 to be fixed by the holding section 53 holding the imaging device 40. The support section 52 and the base 51 are connected via a hinge 512. As a result, as shown in Figure 7, the support section 52 is foldable relative to the base 51.

[0080] A pair of lights 54 are positioned opposite each other across the space on the base 51 where the tray 55 is placed. The pair of lights 54 are configured to shine light toward the tray 55. In the example in Figure 6, the tray 55 is inserted between a pair of walls supporting the pair of lights 54. The tablets placed on the tray 55 are illuminated by light from the lights 54 from almost directly beside them. This reduces the impact of ambient light in the imaging environment on the image quality. Furthermore, the space on the base 51 where the tray 55 is placed is open at both ends in the direction perpendicular to the opposing direction of the pair of lights 54. That is, the indicator unit 52 and the lights 54 are positioned on one side of the surface on the base 51 where the tray 55 is placed. The sides adjacent to the side where the lights 54 are placed are open, with no lights or support units. Therefore, it is easy to slide the dispensing bags placed on the base 51.

[0081] Markers M are provided on the base 51 at positions corresponding to the four corners of the area where the tray 55 is placed. Markers M are marks of a standard color (for example, red). Markers M are provided on a surface parallel to the mounting surface of the tray 55. Multiple markers M are provided, and all markers M have the same shape. Markers M are used for trapezoidal correction of captured images.

[0082] The embodiments described above are examples of the present invention, and the present invention is not limited to these examples. Furthermore, the configurations and processing functions described in the embodiments above can be selected and combined as desired. [Explanation of Symbols]

[0083] 1: Shooting IF unit, 2: Image acquisition unit, 3: Drug name output unit, 4: Non-output drug setting unit, 5: Exclusion character setting unit, 10: Drug identification support system, 20: Identification system

Claims

1. An image acquisition unit that captures an image obtained by photographing a string of characters indicating medication prescribed to an individual, The system includes a drug name output unit that outputs the drug name of at least one drug string that matches a drug name in a drug master from among a group of recognized strings in the image, as a drug candidate identified in the image. The drug name output unit is a drug identification support system that, if the drug name in the drug string is a pre-set non-output drug, does not output the drug name in the drug string as a drug candidate.

2. A drug identification support system according to claim 1, A drug identification support system further comprising a non-output drug setting unit that sets the non-output drugs based on user input.

3. A drug identification support system according to claim 1 or 2, The drug name output unit is a drug identification support system that outputs drug names of drug strings that match drug names in the drug master from the group of strings obtained by removing from the plurality of string groups strings that have a predetermined number of characters including a pre-set exclusion character at the beginning, as drug candidates included in the image.

4. A drug identification support system according to claim 3, A drug identification support system further comprising an exclusion character setting unit that sets the exclusion character based on user input.

5. A method for assisting drug identification performed by a computer, An image acquisition process that involves capturing an image obtained by photographing a string of characters indicating medication prescribed to an individual, The system includes a drug name output step, which outputs the drug name of at least one drug string that matches a drug name in the drug master from among a group of multiple character strings recognized in the image, as a drug candidate identified in the image. In the drug name output step, if the drug name in the drug string is a pre-set non-output drug, the drug name in the drug string is not output as a drug candidate in the drug identification support method.

6. Image acquisition process that captures an image obtained by photographing a string of characters indicating medication prescribed to an individual, The computer is made to perform a drug name output process, which outputs the drug name of at least one drug string that matches a drug name in the drug master from among the multiple groups of strings recognized in the aforementioned image, as a drug candidate identified in the aforementioned image. The program, in the aforementioned drug name output process, does not output the drug name in the drug string as a drug candidate if the drug name is a pre-configured non-output drug.

Images