Method and apparatus for automatically recognizing and inspecting medicines

Abstract

The present invention relates to a method for automatically recognizing and inspecting medicines, the method comprising: a step of acquiring a medicine image including one or more medicines using imaging equipment; a step of recognizing the shape, color, and number of the medicines from the acquired medicine image; a first specification step of specifying a medicine having the same shape and color as the recognized shape and color of the medicine among the shapes and colors of the plurality of medicines stored in the database; a second specification step of, if more than one medicine is specified in the first specification step, recognizing text engraved in the medicines, and specifying a medicine having the same text as recognized text among the text engraved in the plurality of medicines stored in the database; and a step of inspecting whether the total number of recognized medicines and the type of medicine specified in the first specification step or the second specification step match a prescription.

Description

Method and device for automatically recognizing and inspecting drugs

[0001] This patent application claims priority to Republic of Korea Patent Application No. 10-2024-0022358, filed with the Korean Intellectual Property Office on February 16, 2024, the disclosure of which is incorporated herein by reference.

[0002] The present invention relates to a method and device for automatically recognizing and inspecting a drug, and more particularly, to a method and device for automatically recognizing and inspecting a drug for automatically inspecting whether a packaged drug is packaged in the correct type and quantity.

[0003] Medications dispensed at pharmacies or hospitals are packaged in single-dose packages. With recent advances in automated pharmacy dispensing, prescriptions are scanned and automatically packaged into single-dose packages according to the prescription. Pharmacists must also perform a final inspection of medications packaged by automated devices before dispensing them to patients, requiring significant manpower and time.

[0004] In this regard, Korean Patent No. 10-2412874 discloses a video surveillance system and method for a pharmaceutical inspection device. This prior art document recognizes pharmaceutical objects based on pre-learned pharmaceutical images from pharmaceutical images using an object recognition algorithm, and obtains pharmaceutical recognition information on the composition of the pharmaceuticals, including the number of tablets and their form, to determine whether the information matches prescription information.

[0005] The above-mentioned prior literature partially automates dispensing inspection by identifying the number and type of medications. However, given the vast number of medications available, many of which have similar shapes and colors, identifying a medication based solely on its type is extremely difficult, and there is a risk of misidentifying it as a different medication. Furthermore, while medications are sometimes split and prescribed based on the patient's required dosage, the existing literature clearly has a limitation: it cannot identify the type of the split medication.

[0006] Accordingly, there is a need for a method and device for automatically recognizing and inspecting a drug that can improve inspection accuracy by comprehensively considering not only the shape and color of the drug, but also the ingredients of the drug, the roughness of the split cross-section, etc.

[0007] [Prior Art Literature]

[0008] [Patent Document]

[0009] Korean Patent No. 10-2412874

[0010] The purpose of the present invention is to provide a method and device for automatically recognizing and inspecting a drug, which can accurately recognize the type of drug even when the drug has a similar shape and color.

[0011] In addition, the present invention seeks to provide a method and device for automatically recognizing and inspecting a drug that can accurately recognize the type of split drug contained in a drug package.

[0012] In order to achieve the above object, the present invention is characterized in that it comprises a method for automatically recognizing and inspecting a drug, the method comprising the steps of: obtaining a drug image including one or more drugs using an imaging device; recognizing the shape, color, and number of drugs from the obtained drug image; a first specifying step of specifying a drug having the same shape and color as the recognized drug among the shapes and colors of a plurality of drugs stored in a database; a second specifying step of recognizing a letter engraved on the drug and specifying a drug having the same letter as the recognized letter among the letters engraved on a plurality of drugs stored in the database if the drug is not specified as one drug in the first specifying step; and a step of checking whether the total number of recognized drugs and the type of the drug specified in the first specifying step or the second specifying step match a prescription.

[0013] Preferably, the second specific step can recognize the letters engraved on the drug based on optical character recognition (OCR) or a vision-language model.

[0014] Preferably, the method may further include a third specific step of inputting a split drug image included in the acquired drug image into a learned artificial intelligence model to specify the split drug.

[0015] Preferably, the third specific step converts the split drug image into a two-dimensional frequency signal and inputs it into the artificial intelligence model, the artificial intelligence model outputs a two-dimensional frequency signal of tactile sensing data for a cross-section of the split drug, and among the two-dimensional frequency signals of tactile sensing data of a plurality of drugs stored in the database, a drug similar to the two-dimensional frequency signal of tactile sensing data output by the artificial intelligence model can be specified.

[0016] Preferably, the step of checking can check whether the type of drug specified in the first specific step, the second specific step, or the third specific step matches the prescription.

[0017] Preferably, the method may further include a step of training the artificial intelligence model by inputting a first frequency signal obtained by converting a split cross-section image of a specific drug into a two-dimensional frequency signal, and by outputting a second frequency signal obtained by converting tactile sensing data of a split cross-section of a specific drug into a two-dimensional frequency signal.

[0018] Preferably, the method for training the artificial intelligence model may include a preprocessing step of generating a first frequency signal by converting a split cross-section image of a specific drug into a two-dimensional frequency signal and a second frequency signal by converting tactile sensing data of the split cross-section of the specific drug into a two-dimensional frequency signal, generating a plurality of coordinates arranged at regular intervals on the first frequency signal and the second frequency signal, and forming a data pair by corresponding the plurality of coordinates generated on the first frequency signal and the plurality of coordinates generated on the second frequency signal.

[0019] Preferably, the method for training the artificial intelligence model may further include a learning step of training the artificial intelligence model using a first frequency signal and a second frequency signal formed as a data pair.

[0020] Preferably, the step of acquiring the drug image can further acquire a multispectral image of the drug.

[0021] Preferably, the method may include a fourth specific step of analyzing the acquired multispectral image to recognize the type and content of ingredients included in the drug, and specifying a drug having the same type and content of ingredients recognized among the types and contents of ingredients of a plurality of drugs stored in the database.

[0022] Preferably, the step of checking can check whether the type of drug specified in the first specific step, the second specific step, or the fourth specific step matches the prescription.

[0023] In addition, the present invention provides a device for automatically recognizing and inspecting a drug, comprising: a processor including one or more cores; and a memory; wherein the processor acquires a split cross-section image of one or more drugs and tactile sensing data of the split cross-section, generates a first frequency signal by converting the acquired split cross-section image of the drug into a two-dimensional frequency signal, and a second frequency signal by converting the acquired tactile sensing data of the split cross-section of the drug into a two-dimensional frequency signal, generates a plurality of coordinates arranged at regular intervals on the first frequency signal and the second frequency signal, forms a data pair by corresponding the plurality of coordinates generated on the first frequency signal and the plurality of coordinates generated on the second frequency signal, and trains an artificial intelligence model with the first frequency signal and the second frequency signal with which the data pair is formed.

[0024] Preferably, the processor can, when a split cross-sectional image of a specific drug is converted into a two-dimensional frequency signal and input into the artificial intelligence model, cause the artificial intelligence model to output a two-dimensional frequency signal of tactile sensing data of the split cross-section for the specific drug, and can specify a drug similar to the two-dimensional frequency signal of tactile sensing data output by the artificial intelligence model among the two-dimensional frequency signals of tactile sensing data of a plurality of drugs stored in a database.

[0025] The present invention recognizes the type of drug by considering the shape and color of the drug as well as the letters engraved on the drug, so it has the advantage of being able to accurately recognize the type of drug even when the shape and color of the drug are similar.

[0026] In addition, the present invention has the advantage of being able to accurately recognize the type of split drug contained in the drug packaging by inputting an image of the split drug into a learned artificial intelligence model to analyze the split cross-section of the drug and analyzing the components of the drug through a multispectral image.

[0027] Figure 1 shows a flowchart of a method for automatically recognizing and inspecting a drug according to an embodiment of the present invention.

[0028] Figure 2 shows actual implementation screens of the first specific step and the second specific step according to an embodiment of the present invention.

[0029] Figure 3 shows a flowchart of steps for training an artificial intelligence model according to an embodiment of the present invention.

[0030] Figure 4 illustrates a learning data acquisition process for learning an artificial intelligence model according to an embodiment of the present invention.

[0031] Figure 5 illustrates a data pair formation process performed in a preprocessing step according to an embodiment of the present invention.

[0032] Figure 6 illustrates an artificial intelligence model learning process performed in a learning step according to an embodiment of the present invention.

[0033] Figure 7 illustrates a multi-spectral analysis process according to an embodiment of the present invention.

[0034] FIG. 8 is for explaining a method of obtaining drug prescription information. FIG. 8 (a) shows a process of obtaining packaged drug information from EMR, and FIG. 8 (b) shows a process of obtaining packaged drug information by recognizing drug letters on the outer surface of the packaged drug.

[0035] Figure 9 shows a configuration diagram of a device for automatically recognizing and inspecting a drug according to an embodiment of the present invention.

[0036] Figure 10 shows the configuration and operating principle of a system for automatically recognizing and inspecting a drug according to an embodiment of the present invention.

[0037] The present invention relates to a method for automatically recognizing and inspecting a drug, comprising: a step of obtaining a drug image including one or more drugs using an imaging device; a step of recognizing the shape, color, and number of drugs from the obtained drug image; a first specifying step of specifying a drug having the same shape and color as the recognized drug among the shapes and colors of a plurality of drugs stored in a database; a second specifying step of recognizing letters engraved on the drug and specifying a drug having the same letter as the recognized letter among the letters engraved on a plurality of drugs stored in the database if the drug is not specified as one drug in the first specifying step; and a step of checking whether the total number of recognized drugs and the type of the drug specified in the first or second specifying step match a prescription.

[0038] Hereinafter, the present invention will be described in detail with reference to the contents described in the attached drawings. However, the present invention is not limited or restricted by the exemplary embodiments. The same reference numerals in each drawing indicate components that perform substantially the same functions.

[0039] The purpose and effects of the present invention can be naturally understood or made clearer by the following description, and the purpose and effects of the present invention are not limited solely by the following description. Furthermore, in describing the present invention, if a detailed description of known technologies related to the present invention is deemed to unnecessarily obscure the gist of the present invention, such detailed description will be omitted.

[0040] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the description of the invention, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0041] While terms like "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0042] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0043] When interpreting components, even if there is no explicit description, it is interpreted as including the margin of error. When describing temporal relationships, for example, when temporal continuity is described with phrases such as "after," "following," "next to," or "before," this also includes cases where the relationship is not continuous, unless "immediately" or "directly" is used.

[0044] Hereinafter, the technical configuration of the present invention will be described in detail with reference to the attached drawings.

[0045] Fig. 1 is a flowchart of a method for automatically recognizing and inspecting a drug according to an embodiment of the present invention. Referring to Fig. 1, the method for automatically recognizing and inspecting a drug may include a step of acquiring a drug image (S100), a step of recognizing the shape, color, and number of drugs (S200), a first specific step (S300), a second specific step (S400), and an inspection step (S800). The method for automatically recognizing and inspecting a drug may further include any one of a third specific step (S500), a step of training an artificial intelligence model (S600), and a fourth specific step (S700).

[0046] This method for automatically recognizing and inspecting pharmaceuticals considers the shape, color, and text engraved on the pharmaceutical to identify the type of pharmaceutical. This allows for accurate identification of pharmaceutical types even for pharmaceuticals with similar shapes and colors. This method inputs images of fragmented pharmaceuticals into a trained AI model, analyzes the fragmented cross-sections, and analyzes the components of the pharmaceuticals through multispectral imaging. This allows for accurate identification of the type of fragmented pharmaceutical contained within the pharmaceutical packaging.

[0047] Step (S100) of acquiring a drug image may involve acquiring a drug image containing one or more drugs using imaging equipment. The drug image may be of a drug contained within an individually packaged transparent packaging. The drug image may include a complete drug, or may include a drug in a split state (e.g., 1 / 2, 1 / 4, 1 / 8, etc.).

[0048] The step of acquiring a drug image (S100) can acquire a drug image using imaging equipment, and the position and direction of the imaging equipment can be adjusted so that the drug is clearly visible in the drug image. The step of acquiring a drug image (S100) can adjust the position, direction, or brightness of the lighting so that the letters engraved in the drug are clearly visible when capturing the drug image using the imaging equipment.

[0049] The step of acquiring a drug image (S100) can further acquire a multispectral image of the drug. A multispectral image, also called a multispectral image, refers to an image captured from image data belonging to a specific wavelength range of the electromagnetic spectrum. The multispectral image of the drug acquired in the step of acquiring a drug image (S100) can be analyzed in the fourth specific step (S700) to provide the components and content values ​​of the drug. The step of acquiring a drug image (S100) can acquire a multispectral image using a multispectral camera.

[0050] The step of recognizing the shape, color, and number of the medicine (S200) can recognize the shape, color, and number of the medicine from the acquired medicine image. The step of recognizing the shape, color, and number of the medicine (S200) can use a known technology to recognize the shape of the medicine, and in one embodiment, an object recognition algorithm, object recognition artificial intelligence, etc. The step of recognizing the shape, color, and number of the medicine (S200) can use a known technology to recognize the color of the medicine, and in one embodiment, an algorithm for extracting the RGB value of a pixel can be utilized. The step of recognizing the shape, color, and number of the medicine (S200) can use a known technology to recognize the number of the medicine, and in one embodiment, a method of counting the number of shapes of the medicine recognized in the medicine image can be used.

[0051] FIG. 2 illustrates actual implementation screens of the first specific step (S300) and the second specific step (S400) according to an embodiment of the present invention. Referring to FIG. 2, the first specific step (S300) can specify a drug having the same shape and color as a recognized drug among a plurality of drugs stored in a database. The first specific step (S300) can search the database for a drug having the same shape and color as each of one or more recognized drugs. If the same drug is found in the database, the first specific step (S300) can specify the searched drug as the recognized drug.

[0052] The first specific step (S300) distinguishes and recognizes the type of drug based on its shape and color. However, since many drugs have similar shapes and colors among the numerous types of drugs, there are limitations in accurately recognizing the type of drug. Therefore, to overcome the limitations of the first specific step (S300), the recognition rate can be improved through the second specific step (S400), the third specific step (S500), or the fourth specific step (S700), which will be described below.

[0053] The database mentioned in this specification means a storage space that stores all information about a drug, and in one embodiment, the database may store the shape, color, engraved letters, image of a split cross-section of a drug according to the type of drug, data pairs that correspond a plurality of coordinates generated on a first frequency signal and a plurality of coordinates generated on the second frequency signal, and information about the type and content of ingredients.

[0054] The second specific step (S400) can recognize the text engraved on the drug, if it is not identified as a single drug in the first specific step (S300), and identify a drug with the same text recognized among the text engraved on multiple drugs stored in the database. If multiple types of drugs are identified as having the same shape and color in the first specific step (S300), they can be identified as a single drug through the second specific step (S400).

[0055] The second specific step (S400) can recognize the text engraved on the drug based on optical character recognition (OCR) or a vision-language model.

[0056] The letters engraved on the drug are often engraved, making them difficult to see in the drug image. In this case, the second specific step (S400) can control the shooting environment by controlling the position, direction, or brightness of the lighting installed near the imaging equipment, so that the shading of the letters can be clearly seen.

[0057] The third specific step (S500) can input the split drug image included in the acquired drug image into a learned artificial intelligence model to identify the split drug.

[0058] Medications cannot be produced in all possible dosages that can be prescribed to patients. Therefore, pharmacists may split the medication and provide it to patients as needed to match the dosage prescribed by the physician. In this case, when the medication is split and provided to patients, existing automated dispensing inspection systems cannot recognize the type of split medication. In this case, the pharmacist must manually inspect the medication, which is somewhat labor-intensive and time-consuming. The third specific step (S500) according to the present invention utilizes tactile recognition of the roughness and damage characteristics of the split cross-section of the medication, which is one of the methods by which pharmacists determine the type of split medication.

[0059] The third specific step (S500) converts the drug cross-section roughness-enhanced image and tactile sensing data through lighting control into a frequency domain signal, and then creates a combined model through artificial intelligence learning. Then, when an arbitrary split drug cross-section image is captured, it is converted into a frequency domain signal and input into the combined model, thereby restoring the drug tactile sensing data and comparing the similarity with the two-dimensional frequency signal of several drug cross-section tactile sensing data acquired in advance, so that the prescribed drug can be specified.

[0060] Specifically, the third specific step (S500) converts a split drug image into a two-dimensional frequency signal and inputs it into an artificial intelligence model, and the artificial intelligence model outputs a two-dimensional frequency signal of tactile sensing data for a cross-section of the split drug, and among the two-dimensional frequency signals of tactile sensing data of multiple drugs stored in a database, a drug similar to the two-dimensional frequency signal of tactile sensing data output by the artificial intelligence model can be identified.

[0061] The third specific step (S500) can use artificial intelligence to compare the similarity between two-dimensional frequency signals of previously acquired cross-sectional tactile sensing data of various pharmaceutical agents. The artificial intelligence model for comparing similarities can be trained by a person directly comparing the two-dimensional frequency output signal of the restored cross-sectional tactile sensing data of pharmaceutical agents with the frequency signal of the original pharmaceutical agent and labeling the output signal with the original pharmaceutical agent name. When the two-dimensional frequency signal of the restored cross-sectional tactile sensing data of pharmaceutical agents is input to the trained artificial intelligence model, the original pharmaceutical agent name can be output.

[0062] The third specific step (S500) inputs the image of the split drug into an AI model to recognize the degree of splitting of the drug. Here, the degree of splitting refers to the unit of splitting, such as 1 / 2, 1 / 4, or 1 / 8. The reason for recognizing the degree of splitting is that the roughness and damage characteristics of the split cross-section may vary depending on the degree of splitting.

[0063] Figure 3 illustrates a flowchart of steps for training an artificial intelligence model according to an embodiment of the present invention. Referring to Figure 3, the step (S600) for training an artificial intelligence model may include a learning data acquisition step (S610), a preprocessing step (S630), and a learning step (S650).

[0064] The step (S600) of training an artificial intelligence model can train an artificial intelligence model by inputting a first frequency signal obtained by converting a split cross-section image of a specific drug into a two-dimensional frequency signal, and by outputting a second frequency signal obtained by converting tactile sensing data of a split cross-section of a specific drug into a two-dimensional frequency signal.

[0065] FIG. 4 illustrates a learning data acquisition process for learning an artificial intelligence model according to an embodiment of the present invention. Referring to FIG. 4, the learning data acquisition step (S610) may acquire a two-dimensional frequency signal of a split drug image and a two-dimensional frequency signal of tactile sensing data for a cross-section of the split drug. The two-dimensional frequency signal of the split drug image may be acquired through a two-dimensional Fourier transform. In the learning data acquisition step (S610), the unit of the split drug in the split drug image may be 1 / 2, 1 / 4, or 1 / 8, but this is only an example and is not limited thereto.

[0066] Tactile sensing data for a cross-section of a split drug can be sensed by wearing a tactile array sensor on a human finger, and a plurality of sensing data can be stored to form tactile sensing big data for a cross-section of a split drug. A two-dimensional frequency signal of the signal and the tactile sensing data for a cross-section of a split drug can be obtained by performing a two-dimensional Fourier transform on the tactile sensing data.

[0067] Fig. 5 illustrates a data pair formation process performed in a preprocessing step (S630) according to an embodiment of the present invention. Referring to Fig. 5, the preprocessing step (S630) generates a first frequency signal obtained by converting a split cross-section image of a specific drug into a two-dimensional frequency signal and a second frequency signal obtained by converting tactile sensing data of a split cross-section of a specific drug into a two-dimensional frequency signal, generates a plurality of coordinates arranged at regular intervals on the first frequency signal and the second frequency signal, and forms a data pair in which the plurality of coordinates generated on the first frequency signal and the plurality of coordinates generated on the second frequency signal correspond.

[0068] The preprocessing step (S630) can associate the tactile sensation with the cross-section roughness (cross-section damage characteristics) by matching the first frequency signal regarding the cross-section of the split drug with the second frequency signal regarding the tactile sensing data. That is, by training an artificial intelligence model using data pairs formed by matching the first and second frequency signals, the relationship between the cross-section image of the split drug and the tactile sensation of the cross-section can be trained, and through this, the effect of a pharmacist actually using the tactile sensation to distinguish the drug can be generated using only the split drug image. Here, the artificial intelligence model may be a CNN or ViT (Vision Transformer), but this is only an example and is not limited thereto.

[0069] Fig. 6 illustrates an artificial intelligence model learning process performed in a learning step (S650) according to an embodiment of the present invention. Referring to Fig. 6, the learning step (S650) can train the artificial intelligence model using a first frequency signal and a second frequency signal in which data pairs are formed. When a split cross-sectional image of a specific drug is input to the learned artificial intelligence model, a two-dimensional frequency signal of tactile sensing data of the specific drug is output. The two-dimensional frequency signal of the tactile sensing data output during the testing process can be compared with the two-dimensional frequency signal of the actual tactile sensing data to measure an error, and learning can proceed so as to minimize the error.

[0070] Figure 7 illustrates a multispectral analysis process according to an embodiment of the present invention. Referring to Figure 7, the fourth specific step (S700) analyzes the acquired multispectral image to identify the type and content of ingredients contained in the drug, and can identify drugs with the same type and content of ingredients identified among the types and content of ingredients of multiple drugs stored in the database.

[0071] The verification step (S800) can check whether the total number of recognized drugs and the type of drugs specified in the first specific step (S300) or the second specific step (S400) match the prescription. The verification step (S800) can check whether the type of drugs specified in the first specific step (S300), the second specific step (S400), or the third specific step (S500) match the prescription. The verification step (S800) can check whether the type of drugs specified in the first specific step (S300), the second specific step (S400), or the fourth specific step (S700) match the prescription. The inspection step (S800) can inspect whether the type of drug specified in the first specific step (S300), the second specific step (S400), the third specific step (S500), or the fourth specific step (S700) matches the prescription.

[0072] FIG. 8 is for explaining a method of obtaining drug prescription information. FIG. 8 (a) shows a process of obtaining packaged drug information from EMR, and FIG. 8 (b) shows a process of obtaining packaged drug information by recognizing drug letters on the outer surface of the packaged drug.

[0073] Referring to (a) of Fig. 8, packaged drug information can be obtained from EMR (Darwin) to determine whether the type and number of recognized drugs match. Referring to (b) of Fig. 8, packaged drug information can be obtained by recognizing the drug text written on the outer surface of the packaged drug, and whether the type and number of recognized drugs match can be determined.

[0074] FIG. 9 illustrates a configuration diagram of a device (100) for automatically recognizing and inspecting a drug according to an embodiment of the present invention. Referring to FIG. 9, the configuration of the device (100) for automatically recognizing and inspecting a drug illustrated is merely a simplified example.

[0075] A device (100) for automatically recognizing and inspecting a drug may include a processor (110) including one or more cores, a memory (120), and a network (130).

[0076] The processor (110) may be configured with one or more cores, and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. The processor (110) may read a computer program stored in the memory (120) and perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor (110) may perform operations for learning a neural network. The processor (110) may perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating weights of a neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) may process learning of a network function. For example, a CPU and a GPGPU can jointly process network function learning and data classification using network functions. Furthermore, in one embodiment of the present disclosure, processors of multiple computing devices can be jointly used to process network function learning and data classification using network functions. Furthermore, a computer program executed on a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

[0077] The processor (110) can acquire a split cross-section image of one or more drugs and tactile sensing data of the split cross-section. The processor (110) can perform the learning data acquisition step (S610) described above.

[0078] The processor (110) can generate a first frequency signal by converting a split cross-section image of the acquired drug into a two-dimensional frequency signal and a second frequency signal by converting tactile sensing data of the split cross-section of the acquired drug into a two-dimensional frequency signal, generate a plurality of coordinates arranged at regular intervals on the first frequency signal and the second frequency signal, and form a data pair by corresponding the plurality of coordinates generated on the first frequency signal and the plurality of coordinates generated on the second frequency signal. The processor (110) can perform the preprocessing step (S630) described above.

[0079] The processor (110) can train an artificial intelligence model using the first frequency signal and the second frequency signal formed as data pairs. The processor (110) can perform the aforementioned learning step (S650).

[0080] When a split cross-sectional image of a specific drug is converted into a two-dimensional frequency signal and input into the artificial intelligence model, the artificial intelligence model outputs a two-dimensional frequency signal of tactile sensing data of the split cross-section for the specific drug, and among the two-dimensional frequency signals of tactile sensing data of a plurality of drugs stored in a database, the processor (110) can specify a drug similar to the two-dimensional frequency signal of tactile sensing data output by the artificial intelligence model. The processor (110) can perform the third specific step (S500) described above.

[0081] The memory (120) can store any form of information generated or determined by the processor (110) and any form of information received by the network (130).

[0082] The memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may also operate in relation to web storage that performs the storage function of the memory (120) on the internet. The description of the above-described memory is merely an example, and the present disclosure is not limited thereto.

[0083] The network (130) may use any known wired or wireless communication system. The network (130) may receive drug images and the like from related devices or systems.

[0084] The network (130) can transmit and receive information, user interfaces, etc. processed by the processor (110) through communication with other terminals. For example, the network (130) can provide a user interface generated by the processor (100) to a client (e.g., a user terminal). In addition, the network (130) can receive external input from a user authorized as a client and transmit it to the processor (110). At this time, the processor (110) can process operations such as outputting, modifying, changing, and adding information provided through the user interface based on the external input of the user received from the network (130).

[0085] Meanwhile, a device (100) for automatically recognizing and inspecting a drug according to one embodiment of the present disclosure may include a server as a computing system that transmits and receives information through communication with a client. In this case, the client may be any type of terminal capable of accessing the server.

[0086] In a further embodiment, the device (100) for automatically recognizing and inspecting a drug may include any type of terminal that receives data resources generated from any server and performs additional information processing.

[0087] Figure 10 illustrates the configuration and operating principle of a system for automatically recognizing and inspecting a drug according to an embodiment of the present invention. Referring to Figure 10, the system for automatically recognizing and inspecting a drug may include a feeding slot, a spreader & slider, a packaged drug position sensor, a control motor, a lighting device, a camera, a multi-spectral camera, a broadband halogen light, a tension maintenance device, and a computer.

[0088] The feeding slot can be used to insert a packaged medication strip into a medication inspection device. A packaging medication feed detection sensor can be attached to the feeding slot outlet. The system can operate to initiate inspection when the sensor detects packaging medication feeding. The system can switch to an off or power-saving mode when packaging medication feeding is not detected. At this time, a feeding motor, rollers, and a conveyor belt can be provided to enable the packaging medication to be fed. The conveyor belt can be equipped with a self-cleaning device to periodically remove foreign substances.

[0089] The spreader & slider can perform the function of evenly spreading the inserted packaging agent. The spreader & slider shakes the inserted packaging agent strip left and right at regular intervals / cycles, and is equipped with a guide rail whose height continuously decreases in the opposite direction from the direction of insertion, so that the inserted packaging agent can be evenly spread.

[0090] The packaging drug position sensor can sense the position of each individual package of packaging drugs, and the control motor can control each individual package of packaging drugs to be positioned precisely in front of the camera.

[0091] The lighting device can enable the camera to capture images of the medication, and its position, direction, and brightness (intensity) can be controlled. Cameras can be installed on the front and back of the medication, respectively, to capture images of the front and back of the medication. The support plate for the camera-capable portion can be made of transparent glass to capture images of the packaged medication, and the position and direction of each camera can be controlled.

[0092] The tensioning device can maintain tension so that the individually packaged medications can be spread as flat as possible at the camera location. The tensioning device can be positioned at the medication insertion and medication discharge sections, with the camera in the middle.

[0093] While the present invention has been described in detail through representative examples above, those skilled in the art will understand that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims described below but also by all changes or modifications derived from the claims and equivalent concepts.

[0094] The present invention relates to a method and device for automatically recognizing and inspecting a drug, and more particularly, to a method and device for automatically recognizing and inspecting a drug for automatically inspecting whether a packaged drug is packaged in the correct type and quantity.

Claims

1. A method for automatically recognizing and inspecting a drug, A step of obtaining a drug image containing one or more drugs using imaging equipment; A step of recognizing the shape, color, and number of drugs from the acquired drug image; A first specific step of identifying a drug having the same shape and color as the recognized drug among the shapes and colors of multiple drugs stored in a database; If not specified as a single drug in the first specific step, a second specific step of recognizing the letters engraved on the drug and specifying a drug that is identical to the recognized letters among the letters engraved on multiple drugs stored in the database; and A step of checking whether the total number of recognized drugs and the type of drugs specified in the first specific step or the second specific step match the prescription; How to include.

2. In paragraph 1, The above second specific step is, A method for recognizing text engraved on a drug based on optical character recognition (OCR) or a vision-language model.

3. In paragraph 1, A method further comprising a third specific step of inputting a split drug image included in the acquired drug image into a learned artificial intelligence model to specify the split drug.

4. In paragraph 3, The third specific step above is, A method for converting the above-mentioned split drug image into a two-dimensional frequency signal and inputting it into the above-mentioned artificial intelligence model, causing the above-mentioned artificial intelligence model to output a two-dimensional frequency signal of tactile sensing data for a cross-section of the above-mentioned split drug, and specifying a drug similar to the two-dimensional frequency signal of tactile sensing data output by the above-mentioned artificial intelligence model among the two-dimensional frequency signals of tactile sensing data of a plurality of drugs stored in the above-mentioned database.

5. In paragraph 3, The above inspection steps are: A method for checking whether the type of drug specified in the first specific step, the second specific step, or the third specific step matches the prescription.

6. In paragraph 3, A method further comprising a step of training the artificial intelligence model by inputting a first frequency signal, which is a two-dimensional frequency signal converted from a split cross-section image of a specific drug, and outputting a second frequency signal, which is a two-dimensional frequency signal converted from tactile sensing data of a split cross-section of a specific drug.

7. In paragraph 6, The method for training the above artificial intelligence model is as follows: A method comprising a preprocessing step of generating a first frequency signal by converting a split cross-section image of a specific drug into a two-dimensional frequency signal and a second frequency signal by converting tactile sensing data of a split cross-section of a specific drug into a two-dimensional frequency signal, generating a plurality of coordinates arranged at regular intervals on the first frequency signal and the second frequency signal, and forming a data pair by corresponding the plurality of coordinates generated on the first frequency signal and the plurality of coordinates generated on the second frequency signal.

8. In paragraph 7, The method for training the above artificial intelligence model is as follows: A method further comprising a learning step of training the artificial intelligence model using the first frequency signal and the second frequency signal in which the data pairs are formed.

9. In paragraph 1, The step of obtaining the above drug image is: A method for further acquiring multispectral images of the above drug.

10. In paragraph 9, A method comprising: a fourth specific step of analyzing the acquired multispectral image to recognize the type and content of ingredients contained in the drug, and specifying a drug having the same type and content of ingredients recognized among the types and contents of ingredients of multiple drugs stored in the database.

11. In paragraph 10, The above inspection steps are: A method for checking whether the type of drug specified in the first specific step, the second specific step, or the fourth specific step matches the prescription.

12. A device for automatically recognizing and inspecting drugs, a processor comprising one or more cores; and memory; Including, The above processor, Acquire a split cross-section image of one or more drugs and tactile sensing data of the split cross-section, Generating a first frequency signal by converting a split cross-section image of the acquired drug into a two-dimensional frequency signal and a second frequency signal by converting tactile sensing data of the split cross-section of the acquired drug into a two-dimensional frequency signal, generating a plurality of coordinates arranged at regular intervals on the first frequency signal and the second frequency signal, and forming a data pair by corresponding the plurality of coordinates generated on the first frequency signal and the plurality of coordinates generated on the second frequency signal, and A device for training an artificial intelligence model using a first frequency signal and a second frequency signal formed as data pairs.

13. In paragraph 12, The above processor, A device wherein, when a split cross-sectional image of a specific drug is converted into a two-dimensional frequency signal and input into the artificial intelligence model, the artificial intelligence model outputs a two-dimensional frequency signal of tactile sensing data of the split cross-section for the specific drug, and among the two-dimensional frequency signals of tactile sensing data of a plurality of drugs stored in a database, the device specifies a drug similar to the two-dimensional frequency signal of tactile sensing data output by the artificial intelligence model.

Images