Drug inspection system and drug inspection program
The drug inspection system addresses efficiency and accuracy issues by using a windproof member and camera system to stabilize the scale and capture tray contents, ensuring precise drug identification and measurement.
Patent Information
- Application Number
- JP2024066251
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-24
- Filing Date
- 2024-04-16
- Publication Date
- 2024-12-05
AI Technical Summary
The challenge of improving drug inspection efficiency while maintaining measurement accuracy is hindered by the need for a larger scale to accommodate trays, which compromises stability and accuracy due to wind interference on the tray surface.
A drug inspection system with a windshield member positioned outside the load receiver to block wind from affecting the scale, combined with a camera system to capture images of the tray contents, ensuring stability and accuracy by preventing wind interference and allowing for larger tray sizes.
The system enhances inspection efficiency and accuracy by stabilizing the scale against wind interference and enabling larger tray sizes, facilitating precise weight measurement and identification of drugs on the tray.
Smart Images

Figure 2024169316000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a drug inspection system used to inspect drugs dispensed based on prescription data. [Background technology]
[0002] JP 2016-047223 A (Patent Document 1) discloses an inspection support device. This inspection support device includes multiple weighing units and an information reading unit that reads drug identification information from drug packaging materials. Each time the information reading unit reads drug identification information, a process of comparing the weighing result of the weighing unit identified as the weighing unit corresponding to the drug identification information with the prescription data is started. This allows efficient inspection of multiple types of prescription drugs included in the prescription data. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2016-047223 A Summary of the Invention [Problem to be solved by the invention]
[0004] When workers such as pharmacists dispense medicines based on prescriptions, they often check and transport the dispensed medicines by placing them on a tray. The inventors have found that the efficiency of the inspection work can be improved by making it possible to weigh the medicines while they are on the tray. Therefore, from the viewpoint of work efficiency, it is preferable to make the platform of the weighing device larger. However, if the platform is made larger, the stability of the load from the platform to the weighing device may decrease, which may affect the weighing accuracy.
[0005] Therefore, the present application discloses a medicine identification support system and a medicine inspection program that can ensure the efficiency and accuracy of medicine inspection. [Means for solving the problem]
[0006] The drug inspection system in an embodiment of the present invention comprises a weighing scale that measures the weight of the object to be measured, a stand attached to the load receiving part of the weighing scale, a windshield member, and a camera whose imaging range includes the stand attached to the load receiving part of the weighing scale. The windshield member is not in contact with the load receiving portion and is disposed outside the load receiving portion in a position that blocks wind from reaching the underside of the platform. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a medicine inspection system according to this embodiment. [Diagram 2] FIG. 2 is a perspective view showing an example of assembly of the measuring device, the camera, the stand, and the windshield member. [Diagram 3] FIG. 3 is a perspective view showing an example of a state in which the windshield member is attached to the measuring instrument 1 and a state in which it is detached. [Figure 4] FIG. 4 is a cross-sectional view taken along line AA in FIG. [Diagram 5] FIG. 5 is a perspective view showing an example of the configuration of the stand. [Figure 6] FIG. 6 is a diagram for explaining an example of camera arrangement. [Figure 7] FIG. 7 is a diagram showing an example in which a terminal is disposed on one end side in the longitudinal direction of a housing. [Figure 8] FIG. 8 is a diagram for explaining a configuration example of the stand 14. As shown in FIG. [Figure 9] FIG. 9 is a diagram showing a configuration example of an additional windshield cover in this embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the operation of the medicine inspection system shown in FIG. [Figure 11] FIG. 11 is a flowchart showing another operation example of the medicine inspection system shown in FIG. [Figure 12] FIG. 12 is a flowchart showing another operation example of the medicine inspection system shown in FIG. [Figure 13] FIG. 13 is a diagram showing an example of an image captured by the camera 4. As shown in FIG. [Figure 14] FIG. 14 is a diagram showing an example of a marker. [Figure 15] FIG. 15 is a diagram showing an example of an image captured by a camera in the operation example shown in FIG. [Figure 16] FIG. 16 is a diagram showing an example of a detection target range TG1 determined based on markers M1 to M9 in an image. [Figure 17] FIG. 17 is a diagram for explaining an example of preliminary photographing and preliminary detection. [Figure 18] FIG. 18 is a flowchart showing a specific example of the medicine recognition process of FIG. 11 or FIG. [Figure 19] FIG. 19 is a flow chart showing a specific example of the code reading (S25) of FIG. [Figure 20] FIG. 20 is a flow chart showing a specific example of the OCR process (S26) of FIG. [Figure 21] FIG. 21 is a diagram illustrating an example of a learning process for generating a trained model. [Figure 22] FIG. 22 is a diagram for explaining an example of the process of determining the rotation angle in S45 to S47 in FIG. 20(b). [Figure 23] FIG. 23 is a diagram showing an example of the number of intersections between the contour lines and straight lines of a PTP sheet. [Figure 24] FIG. 24 is a diagram showing an example in which a medicine name is displayed superimposed on a medicine area. [Diagram 25] FIG. 25 is a diagram for explaining an example of a process of moving medicine name frames to eliminate overlapping of the medicine name frames. [Figure 26] FIG. 26 shows an example of display before and after the movement of the medicine name frame. [Figure 27] FIG. 27 is a functional block diagram showing an example of the configuration of a medicine inspection system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] The inventors have studied a system for efficiently inspecting a plurality of medicines using one weighing device. In the study, they tried to make the platform attached to the load receiving part of the weighing device larger so that a larger tray could be placed on it. If it is possible to measure medicines on a large tray, the ease and efficiency of inspecting medicines on-site can be improved. However, if the platform is made larger, the measured value of the weighing device may not be stable when a tray containing medicines is placed on the platform. One of the causes of this is thought to be wind hitting the bottom surface of the platform extending in the horizontal direction. Therefore, the inventors have come up with a configuration in which a windshield member is provided to block wind to the bottom surface of the platform extending in the horizontal direction. This configuration ensures the efficiency and accuracy of inspection of medicines. The following embodiments are based on this knowledge.
[0009] (Configuration 1) The drug inspection system in an embodiment of the present invention comprises a weighing scale that measures the weight of the object to be measured, a stand attached to the load receiving part of the weighing scale, a windshield member, and a camera whose imaging range includes the stand attached to the load receiving part of the weighing scale. The windshield member is not in contact with the load receiving portion and is disposed outside the load receiving portion in a position that blocks wind from reaching the underside of the platform.
[0010] According to the above configuration 1, the wind is prevented from reaching the underside of the table by the windbreak member. Therefore, even if the table is extended horizontally, the impact of wind on the weighing accuracy is small. It is possible to ensure accuracy while increasing the horizontal dimension of the table. In addition, since the table is within the camera's shooting range, the medicines on the trays placed on the table are also within the shooting range. As a result, efficiency and accuracy of medicine inspection can be ensured.
[0011] The load receiver of the weighing device is a member that transmits the load of an object to be weighed. The weighing device measures the weight of an object to be measured that is loaded onto the load receiver.
[0012] The windshield member may be provided in a position near an end of the platform to block wind from entering the underside of the platform. The windshield member may include a portion that overlaps with the platform and is positioned below the platform when viewed in the vertical direction, and a portion that is positioned outside the platform when viewed in the vertical direction. This makes it possible to efficiently block wind from entering the underside of the platform.
[0013] The windshield member may be disposed in an area that completely surrounds the load receiving portion when viewed from the vertical direction. This further improves wind blocking performance. For example, the windshield member may be formed of a plate having a hole. In this case, the windshield member may be disposed so that the hole is below the base. The windshield member may also be disposed so that the load receiving portion is inside the hole when viewed from the vertical direction. Furthermore, the windshield member may be disposed so that the inner edge of the windshield member, i.e., the edge of the hole, is outside the load member and inside the base, and the outer edge of the windshield member is outside the base when viewed from the vertical direction.
[0014] (Configuration 2) The windshield member may be spaced apart from the platform and the load receiving portion and may be formed to extend from below the platform toward a side of the platform.
[0015] The platform attached to the load receiving part of the weighing instrument may have a portion located outside the area corresponding to the upper pan of the weighing instrument when viewed from the vertical direction. That is, in at least one direction in the horizontal plane, an end of the platform attached to the load receiving part of the weighing instrument may be located outside the end of the area corresponding to the upper pan of the weighing instrument. In this case, the windshield member may be disposed outside the area of the platform corresponding to the upper pan of the weighing instrument when viewed from the vertical direction. Also, the windshield member may be formed so as to be spaced apart from the platform and extend to the side of the platform from below a position horizontally outside the area of the platform corresponding to the upper pan of the weighing instrument.
[0016] The lower side of the position horizontally outward of the region of the platform that corresponds to the upper pan of the weighing device is a position that is below the platform, overlaps with the platform when viewed vertically, but does not overlap with the region of the platform that corresponds to the upper pan of the weighing device. The configuration in which the windshield member extends to the side of the platform is a configuration in which the windshield member overlaps with at least a portion of the platform when viewed horizontally.
[0017] The platform is attached directly or indirectly to the load receiver. As an example of an indirect attachment, the platform may be attached to another member (e.g., an upper plate, etc.) attached to the load receiver.
[0018] The area corresponding to the upper pan of a weighing instrument is the area of the upper pan attached to the load receiving part of the weighing instrument or the area of the upper pan that is intended to be attached to the load receiving part. For example, if a base is attached to the load receiving part instead of the upper pan, the area of the upper pan that was originally intended to be attached to the load receiving part becomes the area corresponding to the upper pan of the weighing instrument.
[0019] In the above configuration, the platform can be arranged over a wider area in the horizontal direction than the area corresponding to the upper tray. This reduces restrictions on the size and shape of the trays that can be placed on the platform. This allows workers to inspect medicines by placing them in a tray that is suitable for the task, further improving inspection efficiency. From a similar perspective, the "area corresponding to the upper tray of the weighing device" in the above configuration may be replaced with a "weighing device." In this case as well, the platform becomes wider in the horizontal direction, improving inspection efficiency.
[0020] (Configuration 3) In the above configuration 1 or 2, the optical axis of the camera may be arranged at an angle to the vertical direction. This allows an image of the medicines stacked on the tray viewed from an angle to be captured. Therefore, an image in which the number of stacked medicines is easily visible can be obtained. For example, an image in which the number of stacked PTP sheets (medicine sheets) is easily visible can be obtained.
[0021] (Configuration 4) In any of the above configurations 1 to 3, the medicine inspection system may further include a tray identification unit that detects a marker attached to a tray placed on the table from an image captured by the camera and identifies the tray based on the detected marker, and a weight calculation unit that calculates the weight of the medicine on the tray placed on the table using the identification result of the tray identification unit and the measurement value of the weighing device. This makes it possible to measure the weight of the medicine taking into account the weight of the tray on which the medicine is placed. Therefore, the influence of the tray weight on the inspection accuracy is suppressed.
[0022] (Configuration 5) In any of the above configurations 1 to 4, the medicine inspection system may further include a target range determination unit that detects a marker attached to a tray placed on the table from an image captured by the camera and determines a detection target range of medicine in the image based on the detected marker, and a medicine area detection unit that detects a medicine area in the detection target range determined by the target range determination unit. This reduces the effect of a person or object that has moved to a part of the image other than the tray on the detection accuracy of medicine. Furthermore, trays of various shapes and sizes can be used.
[0023] (Configuration 6) In the above configuration 4 or 5, the marker may be configured such that white or black unit areas are arranged without gaps in two mutually orthogonal directions (for example, vertical and horizontal). That is, the marker may be configured such that white or black unit areas are arranged in a matrix. The marker may be configured such that two or more black unit areas are always arranged adjacent to each other. In this case, the marker does not have an arrangement in which the unit areas adjacent to one black unit area vertically and horizontally are all white. This makes it possible to accurately identify the tray or determine the detection target range even if the size of the marker is small. For example, the marker may be configured so that it does not include an arrangement in which the unit areas surrounding one black unit area are all white. Note that the black and white colors in each unit area of the marker do not have to be strictly black and white. Two colors that are different enough to be distinguishable in an image may be the black and white colors of the marker.
[0024] The size of the marker is not particularly limited, but may be, for example, 400 mm. 2 Less than 100mm is preferable 2 Less than 25mm is more preferable. 2 The following is even more preferable: By reducing the size of the marker, the medicines placed on the tray are less likely to overlap the marker, and the degree of freedom in the shape or size of the tray, or the arrangement of the markers on the tray, is increased.
[0025] The tray identification unit or the target range determination unit may detect a marker in a filtered image obtained by performing a filter process for enhancing edges on the image captured by the camera, thereby improving the accuracy of marker detection.
[0026] (Configuration 7) In any of the above configurations 1 to 6, the medicine inspection system may further include an added medicine detection unit that detects the addition of a medicine to the table by at least one of the image captured by the camera and the measurement of the scale when a medicine is added to the table, a weight calculation unit that calculates the weight of the medicine added to the table based on the increase in weight measured by the scale, and a comparison unit that compares the quantity based on the weight of the added medicine with prescription data. This allows the medicines to be placed on a tray one by one, their weights measured, and inspected. Note that the medicines added to the table may be medicines added to a tray placed on the table, as an example.
[0027] In any one of the above configurations 1 to 6, the medicine inspection system includes a medicine recognition unit that recognizes one or more medicines in an image captured by the camera; The drug delivery system may further include a comparison unit that compares the quantity of the one or more drugs based on at least one of the weights of the one or more drugs measured by the weighing scale when the one or more drugs are placed on a table and the images of the one or more drugs taken by the camera with prescription data.
[0028] The drug inspection system may be switchable between a mode in which, for each added drug detected by the added drug detection unit, a quantity based on the weight of each drug measured on a scale is compared with prescription data, and a mode in which, for each added drug detected by the added drug detection unit, one or more drugs on the tray in the image are detected, and a quantity based on at least one of the weights of the one or more drugs measured on the scale for the tray containing the one or more drugs and images of the one or more drugs captured by the camera is compared with prescription data.
[0029] In the above configuration 7, the medicine inspection system may further include a medicine recognition unit that recognizes medicines added to the table in an image captured by the camera. The collation unit may collate the added medicines recognized by the medicine recognition unit with the prescription data. This allows an operator to recognize and calculate the weight of each medicine by simply placing the medicine on a tray. This allows for more efficient inspection.
[0030] The added medicine detection unit may record the area of the medicine detected in the image in a storage unit, whereby the added medicine detection unit can distinguish the next added medicine from the medicines already on the tray.
[0031] In any of the above configurations 1 to 7, the medicine inspection system may further include a collating unit that collates at least one of the recognition result of the medicine in the image captured by the camera and the weight of the medicine measured by the weighing device with prescription data. The recognition result of the medicine in the image may include, for example, at least one of the presence or absence of detection of the medicine, the name of the medicine, and the quantity of the medicine (for example, the number of sheets or the number of tablets, etc.).
[0032] The medicine inspection system in an embodiment of the present invention may include a medicine recognition unit that recognizes medicines by recognizing codes or characters of medicines on a tray placed on a platform in an image captured by a camera having an imaging range covering a platform attached to a load receiving part of a weighing device. In this case, the medicine inspection system may further include a comparison unit that compares the recognition result of the medicine recognized by the medicine recognition unit and the weight of the medicine measured by the weighing device with prescription data.
[0033] The medicine recognition unit may obtain at least one preliminary image captured by the camera during the movement of placing the tray on the table or the movement of placing the medicine on the tray, and may perform preliminary recognition of the code or characters of the medicine in the preliminary image. In this way, when the code or character portion cannot be recognized due to saturation or the like in the image of the medicine on the tray placed on the table, the medicine can be recognized using the code or characters of the medicine preliminarily recognized in the preliminary image. This makes it unnecessary to rearrange and re-photograph the medicine when the code or characters of the medicine in the image cannot be recognized. Therefore, the time required for inspection when the code or characters cannot be recognized can be shortened. The operator can perform the inspection more efficiently.
[0034] The medicine recognition unit may detect a partial image of the medicine PTP sheet area in the image, and execute a process of recognizing a code or character for each of the images obtained by rotating the detected partial image of the PTP sheet area in multiple stages. This makes it possible to recognize the code or character of the medicine PTP with high accuracy and high speed. For example, the medicine recognition unit may execute a process of recognizing a code or character for each of the images obtained by rotating the extracted partial image of the PTP sheet area in multiple stages, which is obtained by applying a filter process to the partial image of the PTP sheet area. This makes it possible to further improve accuracy. The filter process is preferably, for example, a combination of an unsharp mask and a limited histogram equalization process.
[0035] The drug recognition unit may extract at least a partial image of a drug region in the image, determine a rotation angle of the extracted partial image using a trained model that inputs the extracted partial image and outputs a rotation angle for making it easier to recognize characters or codes in the partial image, and execute a process of recognizing codes or characters for an image obtained by rotating the drug region at the determined rotation angle. This can shorten the processing time compared to a case where rotation and recognition processing are repeated within a range of 360 degrees. The trained model can be created, for example, as a neural network model by deep learning using the partial image and the rotation angle as training data.
[0036] The medicine recognition unit may extract a medicine area in the image and execute a code detection process and a character recognition (OCR) process for the medicine area in parallel. In this case, the medicine recognition unit may discard the character recognition result if the code is detected, and may wait for the character recognition result if the code is not detected.
[0037] The medicine inspection system according to an embodiment of the present invention may include a medicine recognition unit that recognizes medicines on a tray placed on a platform in an image captured by a camera having an imaging range covering a platform attached to a load receiving part of a weighing device. In this case, the medicine inspection system may further include a comparison unit that compares the recognition result of the medicine recognized by the medicine recognition unit and the weight of the medicine measured by the weighing device with prescription data.
[0038] The medicine recognition unit may determine the areas of at least the PTP sheet and the pocket in the image captured by the camera using a trained model that outputs the areas of at least the PTP sheet (medicine sheet) and the pocket (medicine storage section) in the input image by segmentation of the image. This allows the medicine area to be recognized with higher accuracy and efficiency.
[0039] The trained model may be, for example, a model that outputs at least the regions of the PTP sheet, pocket, and background in an image. The trained model may be, for example, a model that outputs at least the regions of the PTP sheet, pocket, background, and contour of the PTP sheet in an image. The trained model may be, for example, a model that labels each pixel as at least one of the PTP sheet, pocket, and background by semantic segmentation. Note that the trained model is not limited to a semantic segmentation model, and may be, for example, an instance segmentation or panoptic segmentation model.
[0040] The medicine recognition unit may obtain an image including a plurality of overlapping PTP sheets in the tray, the image being taken using the autofocus of the camera, and may determine the area of one of the overlapping PTP sheets by detecting the edge of the most focused PTP sheet among the overlapping PTP sheets. This allows, for example, the area of the topmost PTP sheet among the overlapping PTP sheets to be detected. This increases the detection accuracy of the PTP sheets. This also makes it easier to calculate the weight of each PTP sheet. For example, the medicine recognition unit may detect the edge of the most focused PTP sheet among the overlapping PTP sheets as the outline of the fractional sheet.
[0041] The medicine recognition unit may detect the contour of the PTP sheet in the image, and determine the number of PTP sheets based on the number of intersections between a reference straight line and the detected contour of the PTP sheet. This allows the number of PTP sheets to be identified based on the image. The reference straight line may be, for example, a straight line passing through a minimum circumscribing rectangle of the area of the PTP sheet. There may be multiple reference straight lines. In this case, the number is determined based on the number of intersections between each of the multiple reference straight lines and the contour of the PTP sheet. This allows the number to be identified with greater accuracy. The multiple reference straight lines may include, for example, at least one of two straight lines that are perpendicular to each other and two or more parallel straight lines.
[0042] In any of the above configurations 1 to 7, the platform may be configured so that the weight per unit of the portion that overlaps with the area corresponding to the upper plate of the weighing device when viewed from the vertical direction is greater than the weight per unit of the portion that does not overlap with the area corresponding to the upper plate of the weighing device. This makes it difficult for the platform to tilt even if a drug is placed on the portion of the platform that does not overlap with the area corresponding to the upper plate of the weighing device. Therefore, it is possible to ensure the accuracy of weighing while widening the range in which the drug can be placed. As a result, it becomes easier to ensure the efficiency and accuracy of drug inspection.
[0043] For example, a counterweight may be provided in a portion that overlaps with the area corresponding to the upper plate of the weighing device when viewed from the vertical direction of the platform. Also, at least one of a thin-walled portion having a thickness thinner than that of the overlapping portion and a hole may be provided in a portion that does not overlap with the area corresponding to the upper plate of the weighing device when viewed from the vertical direction of the platform.
[0044] In any of the above configurations 1 to 7, the medicine inspection system may include a block that supports the bottom surface of the measuring device at at least three points, and a base fixed to the block. The block may be provided with legs that contact the installation surface. This allows the measuring device to be stably supported. For example, distortion of the base or the like does not affect the support structure of the measuring device. This makes it easier to ensure measuring accuracy. For example, a camera support member that supports the camera may be attached to the base. Or, a housing that houses the measuring device may be attached to the base. For example, the windshield member may be attached to the housing.
[0045] In any of the above configurations 2, and 4 to 7, the medicine inspection system may omit the windshield member.
[0046] A drug inspection program in an embodiment of the present invention causes a computer connected to a camera whose imaging range includes a weighing scale and a platform attached to the load receiving part of the weighing scale to perform the following processes: detect a marker on a tray placed on the platform from an image captured by the camera; identify the tray based on the detected marker; and calculate the weight of the drug on the tray placed on the platform using the tray identification result and the weighing value measured by the weighing scale.
[0047] A drug inspection program in an embodiment of the present invention causes a computer connected to a camera whose imaging range includes a weighing scale and a stand attached to the load receiving part of the weighing scale to execute the following processes: detecting markers attached to a tray placed on the stand from an image captured by the camera, determining a detection range for drugs in the image based on the detected markers, and detecting a drug area within the determined detection range.
[0048] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Embodiment] (System configuration example) 1 is a diagram showing an example configuration of a medicine inspection system 10 of this embodiment. The medicine inspection system 10 includes a weighing scale 1, a stand 2, a windshield member 3, a camera 4, a terminal 5, and a reader 6. In the medicine inspection system 10, a computer in the terminal 5 determines whether information about the medicine to be inspected obtained from at least one of the weighing scale 1, the camera 4, the reader 6, and user input matches the medicine indicated by the prescription data.
[0049] Weighing scale 1 is a weight measuring device that measures the weight of an object to be measured. Platform 2 is attached to the load receiving portion of weighing scale 1. Windshield member 3 is attached to weighing scale 1 in a state where no load is applied to the load receiving portion. In other words, windshield member 3 is attached to a position spaced apart from platform 2 and the load receiving portion. Windshield member 3 is formed extending from below platform 2 to the side of platform 2. Camera 4 is supported by camera support portion 4a at a position that includes platform 2 attached to weighing scale 1 in its shooting range. In the example of FIG. 1, platform 2 and windshield member 3 are detachable from weighing scale 1.
[0050] The terminal 5 includes a computer 5a, a display 5b, and an input device 5c. The computer 5a is connected to the weighing device 1, the camera 4, and the reader 6 by wire or wirelessly and can communicate with them. The computer 5a controls the weighing device 1, the camera 4, and the reader 6 and processes data acquired from them. The data processed by the computer 5a is stored in a storage unit 8. The storage unit 8 may be a storage device built into the computer 5a or an external storage device accessible by the computer 5a. The display 5b is an example of an output device to which information from the computer 5a is output. The input device 5c accepts input from a user to the computer 5a. The input device 5c may be, for example, a touch panel, a mouse, a keyboard, a button, or another operation unit. The terminal 5 may be, for example, a tablet terminal, a laptop computer, a desktop computer, a smartphone, or a dedicated computer.
[0051] The medicine inspection system 10 includes a tray identification unit 51, a weight calculation unit 52, a target range determination unit 53, an added medicine detection unit 54, a medicine recognition unit 55, a collation unit 56, and a control unit 57 as functional units realized by the computer 5a. The medicine recognition unit 55 includes a medicine area detection unit 251. These functional units can be realized by one or more computers executing a predetermined program. Such programs and non-transitory storage media storing the programs are also included in the embodiments of the present invention. Note that the computer constituting these functional units is not limited to only the terminal 5. For example, at least a part of these functional units may be constituted by one or more computers connected to the terminal 5 via a network.
[0052] The reader 6 is an information reader that reads medicine identification information from the medicine packaging material. The reader 6 is, for example, a code reader. In this case, the reader 6 can read identification information such as a one-dimensional code such as a barcode (JAN, GS1) or a two-dimensional code such as a QR code (registered trademark).
[0053] (Example of device configuration) FIG. 2 is a perspective view showing an example of assembly of a weighing device 1, a camera 4, a base 2, and a windshield member 3. In the example of FIG. 2, the weighing device 1 has a housing 1b, a load receiving portion 1a provided on the upper surface of the housing 1b, and an upper pan 1d attached on the load receiving portion 1a. The load receiving portion 1a is movable with respect to the housing 1b. The load receiving portion 1a receives the load of the object to be measured and transmits the load to a measuring mechanism built into the housing 1b. The measuring mechanism can be, for example, an electromagnetic type or a load cell type. The weighing device 1 can be, for example, an electronic balance.
[0054] The upper plate 1d is removed, and instead, the base 2 is attached directly to the load receiving portion 1a. The base 2 is plate-shaped. A connection portion 2c that is connected to the load receiving portion 1a is provided on the underside of the base 2 at a position corresponding to the load receiving portion 1a. The connection portion 2c is formed, for example, as a recess that fits into the load receiving portion 1a. The base 2 may also be attached to the upper plate 1d. In this case, the connection portion 2c is shaped to fit into the upper plate 1d.
[0055] The windshield member 3 is a plate having a hole 3a. The windshield member 3 is attached to the weighing device 1 without applying a load to the load receiving portion 1a. In the example of FIG. 2, the windshield member 3 is placed on the outside of the area corresponding to the upper plate 1d on the upper surface of the housing 1b of the weighing device 1. When viewed from the vertical direction, the hole 3a of the windshield member 3 is larger than the area corresponding to the upper plate 1d. The connection portion 2c of the base 2 is attached to the load receiving portion 1a through the hole 3a of the windshield member 3. When viewed from the vertical direction, the outer edge of the windshield member 3 is located outside the outer edge of the base 2. That is, when viewed from the vertical direction, the windshield member 3 is disposed at a position overlapping the second outer edge. The windshield member 3 also has a flange portion 3b extending upward outside the base 2 in the horizontal direction.
[0056] In the example of FIG. 2, the base 2 attached to the weighing device 1 is arranged in a range wider than the area corresponding to the upper tray 1d when viewed vertically. Also, when viewed vertically, at least a part of the base 2 is located outside the weighing device 1. In this way, the mounting surface of the base 2 can be made wider. Therefore, the size and shape of the tray 7 to be placed on the base 2 can be made more flexible. Therefore, the user can use a tray 7 suitable for dispensing and inspection. This improves the inspection efficiency. Also, when the horizontal range of the base 2 is increased, the influence of wind on the bottom surface of the base 2 becomes an issue, but since the windshield member 3 is arranged from the bottom surface of the base 2 to the side, the influence of wind is suppressed. Therefore, the efficiency and accuracy of the inspection of medicines can be ensured.
[0057] In the example of FIG. 2, the weighing device 1 has legs 1c provided on the bottom surface of the housing 1b. Legs 1c are supported by a block 11. Block 11 supports legs 1c so that the weighing device 1 is horizontal. Legs 11a are provided on the bottom surface of block 11. Legs 11a of block 11 contact the installation surface. A base 12 is fixed to block 11. Base 12 is fixed to block 11 by, for example, a fastener (not shown). Legs 11a of block 11 pass through holes 12a in base 12. Although not shown in FIG. 2, a covering housing that stores the weighing device 1 may be attached to base 12.
[0058] FIG. 3(a) is a perspective view showing an example of a state in which the base 2 and the windshield member 3 are attached to the weighing device 1. FIG. 3(b) is a perspective view showing a state in which the windshield member 3 is omitted from FIG. 3(a). In the state of FIG. 3(b), wind hits the bottom surface of the base 2 directly. The influence of the wind may increase the variation in the measured value of the weighing device 1, and the measurement accuracy may decrease. The influence of the wind may slow down the measurement speed of the weighing device 1. In response to this, as shown in FIG. 3(a), by providing the windshield member 3, the wind blowing up from under the base 2 is blocked by the windshield member 3. Therefore, the effect of the wind on the measurement accuracy and speed is reduced. In the example of FIG. 3, a housing 13 is provided to house the weighing device 1. The housing 13 covers the top surface of the weighing device 1.
[0059] As an example, it is assumed that the medicine inspection system 10 is used in an environment where the air current of an indoor air conditioner or the like flows from top to bottom. In such an environment, a phenomenon may occur in which the air current bounces off a placement surface such as a table surface on which the weighing device 1 of the medicine inspection system 10 is placed, and the air current flows toward the lower surface of the platform 2 attached to the load receiving portion 1a. In this case, for example, as shown in FIG. 3(b), if there is nothing blocking the air current flowing from the placement surface toward the lower surface of the platform 2, an unstable air current from below will always continue to hit the bottom surface of the tray, and the measurement value will be unstable. To avoid this situation, in this embodiment, a windshield member 3 is provided. The windshield member 3 will receive the rebound air current. Since the windshield member 3 is not in contact with the load receiving portion 1a, the measurement value of the weighing device 1 is prevented from being destabilized by the air current.
[0060] FIG. 4 is a cross-sectional view of line AA in FIG. 3(a). As shown in FIG. 4, the base 2 is formed extending horizontally outside the area corresponding to the upper plate 1d of the weighing device 1. The windshield member 3 is disposed below the horizontally outer portion of the area corresponding to the upper plate 1d of the base 2. The windshield member 3 extends horizontally outward from the bottom of the base 2 and reaches the outer side of the horizontal end of the base 2. Thus, the windshield member 3 is formed extending from at least a part of the area outside the area corresponding to the upper plate 1d of the weighing device 1 of the base 2 when viewed vertically to the outside of the base 2. That is, the area where the windshield member 3 is disposed when viewed vertically includes at least a part of the area overlapping with the base 2 outside the area corresponding to the upper plate 1d of the weighing device 1, and an area extending from this at least a part of the area to the outside of the base 2.
[0061] In the example of FIG. 4, the windshield member 3 has a portion that extends upward on the horizontal outer side of the platform 2, i.e., a flange portion 3b. The flange portion 3b is the end portion of the windshield member 3. The flange portion 3b overlaps with at least a portion of the platform 2 when viewed horizontally. In the example of FIG. 4, the upper end of the flange portion 3b, i.e., the upper end of the windshield member 3, is higher than the upper end of the platform 2. The windshield member 3 is not positioned above the platform 2. In other words, the windshield member 3 has a shape that opens upward. This makes it easier to place the tray 7 on the platform 2 while preventing wind from blowing into the platform 2, which would affect weighing.
[0062] In the example of FIG. 4, the windshield member 3 is attached to the upper surface of the housing 1b of the weighing device 1. The windshield member 3 is attached at a position separated from both the base 2 and the load receiving portion 1a. As a result, the windshield member 3 is attached at a location separated from the load transmission path of the weighing device 1, i.e., at a location separated from the weighing mechanism (balance) of the weighing device 1. In the example of FIG. 4, the windshield member 3 is supported by the housing 1b and the housing 13. The windshield member 3 may be attached so as to be supported by at least one of the housing 1b and the housing 13. Alternatively, the windshield member 3 may be provided independently, separated from both the housing 1b and the housing 13.
[0063] In the example of FIG. 4, the weighing device 1 is placed on the block 11, and the legs 11a on the bottom surface of the block 11 are in contact with the installation surface (floor). The plate-shaped base 12 is fixed to the block 11. The weighing device 1 is supported on the installation surface via the block 11, not via the base 12. This reduces the effect of distortion of the base 12 on the measurement of the weighing device 1. For example, in the case of a configuration in which the weighing device 1 is placed on the base 12, if the base 12 is distorted, the weighing device 1 will rattle and become unstable relative to the installation surface. This may make measurement impossible or may take a long time to measure. By configuring the weighing device 1 to be supported by the block 11 as in FIG. 4, the weighing device 1 can be stably supported.
[0064] Block 11 may be made of a metal such as aluminum. Legs 11a of block 11 may be made of a resin such as rubber. For example, base 12 may be fixed to block 11 by a fastener such as a bolt that passes through base 12 and reaches block 11. In the example of FIG. 4, a recess is provided on the top surface of block 11. Legs 1c of measuring device 1 are placed in the recess in the top surface of block 11. This improves stability.
[0065] The blocks 11 are provided corresponding to each of the multiple legs 1c on the bottom surface of the weighing device 1. In this example, four blocks 11 are provided corresponding to the four legs 1c of the weighing device 1, respectively. The configuration of the blocks 11 is not limited to this example. As a modified example, a block common to the multiple legs 1c may be provided. The blocks support at least three points on the bottom surface of the weighing device.
[0066] FIG. 5 is a perspective view showing an example of the configuration of the table 2. FIG. 5(a) is a perspective view of the table 2 seen from the top (front surface), and FIG. 5(b) is a perspective view of the table seen from the bottom (back surface). In the example of FIG. 5, a counterweight 2b is provided in the horizontal center of the table 2, and a hole 2a is provided outside the center. This makes the weight per unit at the center of the table larger than the weight per unit at the other parts of the table. The counterweight 2b may be provided in an area that overlaps with an area corresponding to the upper plate of the weighing device 1 when the table 2 is attached to the weighing device 1. Even if a drug is placed near the edge of the table 2, the counterweight 2b causes the center of gravity of the drug and the table 2 to overlap with the area corresponding to the upper plate. This prevents the tray 7 from tilting when a drug is placed near the edge of the tray 7 placed on the table 2. This stabilizes the weight measurement. That is, the size of the table 2 can be increased to improve the inspection efficiency and ensure the weighing accuracy.
[0067] FIG. 6 is a diagram for explaining an example of the arrangement of the camera 4. FIG. 6(a) is a front view of the device including the camera 4, the weighing device 1, and the stand 2, and FIG. 6(b) is a side view of the device. In the example of FIG. 6, the camera 4 is supported by the camera support part 4a with the optical axis C1 of the camera 4 tilted with respect to the vertical direction. This allows the medicines on the tray 7 placed on the stand 2 to be photographed obliquely. This allows an image that makes it easy to see, for example, the number of stacked PTP sheets. From this viewpoint, it is preferable that the angle θ1 of the optical axis C1 of the camera 4 with respect to the vertical direction is 25 degrees≦θ1≦65 degrees.
[0068] Furthermore, the angles of view Gh and Gv of the camera 4 are set so that the entirety of the platform 2 falls within the shooting range of the camera 4 when the camera 4 is supported by the camera support portion 4a. The lower limits of the horizontal angle of view Gh and the vertical angle of view Gv of the camera 4 can be set to, for example, 50 degrees (Gh≧50 degrees, Gv≧50 degrees). It is preferable that the horizontal angle of view Gh is, for example, 100 degrees or more (Gh≧100 degrees).
[0069] The left-right direction in Fig. 6(a) is the longitudinal direction of the housing 13, and the left-right direction in Fig. 6(b) is the lateral direction of the housing 13. In the example of Fig. 6(b), the terminal 5 is disposed at one end side in the lateral direction of the housing 13 while being supported by the stand 14. The stand 14 supports the terminal 5 with the display screen of the terminal 5 tilted with respect to the horizontal plane.
[0070] 7 is a diagram showing an example in which the terminal 5 is disposed on one end side of the housing 13 in the longitudinal direction. In this case, the inclination of the display screen of the terminal 5 supported by the stand 14 differs from the case shown in FIG. 6(b). The upper surface near one end of the longitudinal direction of the housing 13 has an inclined portion that slopes downward toward the end. The display screen of the terminal 5 supported by the stand 14 overlaps the inclined portion in the vertical direction and is inclined along the inclined portion. This makes it possible to reduce the horizontal space occupied by the terminal 5 and the housing 13.
[0071] FIG. 8 is a diagram for explaining a configuration example of the stand 14. FIGS. 8(a) and 8(b) show a configuration example of the stand that holds the terminal 5 in a first state. FIGS. 8(c) and 8(d) show a configuration example of the stand that holds the terminal 5 in a second state different from the first state. In the example of FIG. 8, the stand 14 has a holding part 14b that holds the terminal 5 and a support part 14a that supports the holding part 14b. The angle of the holding part 14b relative to the support part 14a is adjustable. This makes it possible to adjust the angle at which the stand 14 holds the terminal 5. Therefore, the terminal 5 can be arranged efficiently. Also, in the example of FIGS. 8(c) and 8(d), an attachment 14c is attached to the holding part 14b of the stand 14. By attaching and detaching the attachment 14c, the range of angles at which the terminal 5 can be held can be expanded.
[0072] FIG. 9 is a diagram showing an example of the configuration of an additional windshield cover in this embodiment. The windshield cover is an external windshield cover that covers the entire weighing device 1, the camera 4, the base 2, and the housing 13. The windshield cover includes a main body 15 that is a combination of panels that open at the front and bottom and form the sides, top, and back, and guides 15A and 15B that are panels that extend from the side panels toward the front. The front edge of the side panel of the main body 15 has a recess 15c. The recess 15c is provided at a height corresponding to the base 2 of the weighing device 1. The recess 15c ensures the movement of objects or hands when the user places the tray 7 on the base 2 or places medicines on the tray 7 placed on the base 2. The guides 15A and 15B are provided at positions that overlap the recess 15c when viewed from the side. This prevents wind from entering the weighing device 1 from the recess 15c. With this windshield cover, the efficiency of the operation for placing the medicine can be improved and the influence of wind can be reduced to ensure the measurement accuracy. In the example of Fig. 9, the guides are provided on the panels on both sides of the main body, but the guides may be provided on at least one of the side panels on both sides.
[0073] In the example of FIG. 9, the reader 6 is provided on the ceiling of the windshield cover. The reader 6 is arranged so that the lead portion faces downward. The reader 6 reads the code of the medicine located below the reader 6 inside the windshield cover. This allows, for example, the user to read the code on the PTP sheet with the reader 6, which is a barcode reader, in the process of placing the PTP sheet on the tray 7. The user can place the PTP sheet on the tray 7 and have the reader 6 read the code in a single action.
[0074] (Example of operation) FIG. 10 is a flowchart showing an example of the operation of the medicine inspection system shown in FIG. 1. FIG. 10 shows an example of the operation of sequentially inspecting medicines dispensed based on a prescription, one by one. A user, who is an operator, first places a tray 7 on a stand 2 attached to a weighing scale 1. The medicine inspection system detects that the tray 7 has been placed (S1). The computer of the terminal 5 can detect that the tray 7 has been placed based on at least one of the weight measured by the weighing scale 1 and the image captured by the camera 4.
[0075] In S1, the tray identification unit 51 may detect a marker attached to the tray 7 in the image captured by the camera 4, and identify the tray based on the marker. The tray identification unit 51 may, for example, identify the individual tray based on the marker. In this case, it is preferable that the marker is, for example, a barcode, a two-dimensional code, or an Aruco marker, whose pattern indicates information. In this case, the information indicated by the marker can be used as an identifier for identifying the individual tray.
[0076] Weight calculation unit 52 can calculate the weight of the medicine on the tray using the identification result of tray identification unit 51 and the measurement value of weighing device 1. In this case, tray-specific information used for weight measurement, such as the tray weight and other tray-specific parameters related to weight measurement, can be recorded in memory unit 8 in association with the tray identifier. For example, as described below, weight calculation unit 52 may calculate the weight of the medicine on the tray using tray-specific information corresponding to a tray identifier identified by a mark included in an image captured by camera 4 and the measurement value of weighing device 1.
[0077] Furthermore, tray identification unit 51 may identify the tray area in the image based on the position of the marker in the image captured by camera 4. In this case, as described below, target range determination unit 53 can determine the tray area in the image as the detection target range for the medicine. Medicine area detection unit 551 of medicine recognition unit 55 detects the medicine area in the detection target range.
[0078] The computer accepts the specification of prescription data from the user (S2). In S2, for example, the computer accepts the specification of prescription data by displaying a screen for accepting the specification of prescription data on the display 5b of the terminal 5. The user inputs the specification of the prescription data to be inspected via the input device 5c such as a touch panel.
[0079] The user causes the reader 6 to read the medicine identification information (e.g., JAN code, RSS code, YJ code, or GS1 code, etc.) printed on the packaging material of one of the medicines dispensed based on the prescription indicated by the prescription data. The medicine identification information of one medicine is read by the reader 6 (S3). The packaging material of the medicine may be, for example, a packaging paper, a PTP sheet, or a medicine container.
[0080] In S4, the comparison unit compares the drug indicated by the drug identification information read in S3 with the drug indicated by the prescription data specified in S2. If it is determined that they do not match (NO in S4), the computer outputs an alarm to the user (S5). The alarm output may be, for example, an audio output and a display. The alarm may also include a notification that the read drug does not match the prescription.
[0081] If the answer is YES in S4, the computer waits for the drug to be added to the tray 7. Here, for example, a message may be outputted to prompt the user to put the drug indicated by the drug identification information read in S3 into the tray 7.
[0082] In S6, the added medicine detection unit 54 detects that medicine has been added to the tray 7 (i.e., platform 2) based on at least one of the weight measured by the scale 1 and the image captured by the camera 4. For example, the added medicine may be detected by detecting an increase in the weight measured by the scale 1 by more than a predetermined amount, or by detecting a new medicine area in the image captured by the camera 4, or by detecting both of these.
[0083] When the addition of a medicine to the tray 7 is detected (YES in S6), the weight calculation unit 52 calculates the weight of the added medicine based on the measurement value of the weighing device 1 (S7). For example, the weight of the added medicine can be calculated based on the increase in weight measured by the weighing device 1 within a predetermined time from the time when a new medicine area is detected in the image from the camera 4. As one example, the weight of the medicine may be calculated by subtracting the weight of the packaging material from the measured increase in weight. The weight of the added medicine may also be calculated based on the weight measured by the weighing device 1 minus the weight of the tray 7 and the weight of medicine other than the added medicine already on the tray 7.
[0084] In S8, the medicine recognition unit 55 determines the quantity of the added medicine based on the weight calculated in S7. For example, the calculated weight may be converted into the number of tablets or sheets. This conversion can be performed using the weight per unit quantity of the medicine (e.g., one tablet, one sheet). The weight per unit quantity may be obtained, for example, from a medicine database. Note that, when the quantity of the medicine indicated in the prescription data is expressed by weight, the weight calculated in S7 may be used as the quantity of the medicine.
[0085] In S9, the collation unit 56 collates the quantity of the drug determined in S7 with the quantity of the drug indicated in the prescription data, and determines whether they match. For example, if the difference between the quantity of the drug determined in S7 and the quantity indicated in the prescription data is within a predetermined range, it may be determined that they match. For example, in the collation process, when the weight of the drug determined in S7 is compared with the weight of the drug indicated in the prescription data, it can be determined that they match if the difference between these weights is within an allowable error range. The allowable error range may be determined according to the weight of the drug, for example. If the quantities do not match in the collation (NO in S9), the computer outputs an alarm (S5). The alarm may include a notification that the quantity of the drug does not match the prescription.
[0086] If there are any medicines that have not yet been read among the medicines indicated by the prescription data (YES in S10), the inspection process of S3 to S9 described above is repeatedly executed. In this case, a message may be outputted to prompt the user to have the reader 6 read the next medicine and add it to the tray 7. The inspection process of S3 to S9 is executed for all medicines indicated by the prescription data. As a result, the user can repeat the operation of having the reader 6 read one medicine and add it to the tray 7 for all dispensed medicines, thereby executing the inspection process sequentially for each medicine.
[0087] When the inspection process is completed for all medicines indicated by the prescription data (NO in S10), the medicines on the tray 7 are photographed by the camera 4 and recorded as an evidence image in the memory unit 8 (S11). The inspection result is then output (S12). The inspection result indicates whether or not the medicines and the quantities of the medicines recognized by the medicine inspection system 10 match the information in the prescription data. The form of output is not particularly limited, and may be, for example, at least one of display on the display 5b, recording in the memory unit 8, and transmission.
[0088] FIG. 11 is a flowchart showing another example of the operation of the medicine inspection system shown in FIG. 1. FIG. 11 shows an example of the operation of inspecting medicines dispensed based on a prescription one by one in sequence. In the example of FIG. 11, medicine identification information is read from an image captured by camera 4, not by a reader. Therefore, whereas the example of FIG. 10 involves two steps, namely, holding the medicine over reader 6 and placing it on tray 7 (weighing device 1), in the example of FIG. 11, the user's work is reduced to just placing the medicine on tray 7 (weighing device 1).
[0089] The processes of S1, S2, S11, and S12 in Fig. 11 can be similar to S1 and S2 in Fig. 10. After S2, the computer waits for one drug to be added to the tray 7. Here, for example, a message may be output to prompt the user to put one drug into the tray 7.
[0090] In S3-2, the computer detects that a medicine has been added to the tray 7 (i.e., the table 2). This detection can be performed in the same manner as the detection in S6 of FIG. 10. When the addition of a medicine to the tray 7 is detected (YES in S3-2), calculation of the weight of the added medicine (S7) and recognition of the added medicine from the image captured by the camera 4 (S3-3) are started. That is, the calculation of the weight of the added medicine and the recognition process of the medicine in the image are performed in parallel. The processes in S7 to S9 of FIG. 11 can be performed in the same manner as S7 to S9 of FIG. 10.
[0091] In S3-3, for example, the medicine recognition unit 55 executes reading of medicine identification information in an image including the added medicine captured by the camera 4. The medicine identification information may be, for example, a JAN code, an RSS code, a YJ code, or a GS1 code, or characters representing the medicine name. In this case, the medicine recognition unit 55 executes a process of reading the medicine identification information in the form of a code or characters written on the packaging material of the medicine in the image. This recognizes the added medicine. The process of comparing the recognized medicine with prescription data (S4) and outputting an alarm if the answer is NO in S4 (S5) can be performed in the same manner as S4 and S5 in FIG. 10.
[0092] The processes of S3-2, S3-3, and S4 to S9 in FIG. 11 are executed for all medicines indicated in the prescription data. As a result, the user repeats the operation of adding one medicine to the tray 7 (table 2) for all dispensed medicines, and the inspection process is executed sequentially for each medicine. This eliminates the need for the user to input the medicine into the reader, reducing the number of steps required. Also, since the recognition of the medicine in the image and the weight measurement are executed in parallel, the processing time can be shortened.
[0093] In the example of FIG. 11, when an added drug is recognized in the image in S3-3, the area of the recognized drug in the image (drug area) may be recorded in the storage unit 8 as a recognized area. By recording the recognized area in this way, when a new drug is added, it becomes easier to distinguish between the area of the newly added drug and the area of a drug that has already been recognized. For example, the added drug detection unit 54 can detect the addition of a drug by detecting a drug area in an area other than the recognized area in the image.
[0094] FIG. 12 is a flowchart showing another example of the operation of the medicine inspection system shown in FIG. 1. FIG. 12 is an example of the operation of inspecting one or more medicines dispensed based on a prescription all at once. In the example of FIG. 12, a user places a tray 7 containing all the medicines based on one prescription on a weighing scale 1 (table 2). The processes of S1, S2, and S12 in FIG. 12 can be performed in the same manner as S1, S2, and S12 in FIG. 10. In S3-4, the medicine recognition unit 55 recognizes all of one or more medicines included in the image of the tray 7 captured by the camera 4. The recognition process of each medicine can be performed in the same manner as the recognition process of one medicine in S3-3 in FIG. 11. The processes of S4 and S5 can be performed in the same manner as S4 and S5 in FIG. 10.
[0095] In S8-2, the medicine recognition unit 55 determines the quantity of the one or more medicines based on at least one of the weights of the one or more medicines measured by the weighing device 1 and the image captured by the camera 4 that includes the one or more medicines. The weight calculation unit 52 may calculate the total weight of the one or more medicines based on the weight measured by the weighing device 1 when the one or more medicines recognized in S3-4 are placed on the tray 7 (base 2). The medicine recognition unit 55 may determine this total weight as the quantity of one or more medicines. The total weight of the one or more medicines can be calculated based on the weight obtained by subtracting the weight of the tray 7 from the measured value of the weighing device 1 when the tray 7 containing one or more medicines is placed on the base 2. The weight of the tray 7 can be determined based on the marker of the tray 7 in the image. For example, the weight of the tray can be recorded in advance in the storage unit 8 in association with the identifier of the tray. The weight calculation unit 52 can refer to the storage unit 8 to determine the weight of the tray corresponding to the identification indicated by the marker.
[0096] Alternatively, in S8-2, the medicine recognition unit 55 may execute a process of recognizing the number of PTP sheets or the number of tablets in the image (a specific example will be described later).
[0097] The collation unit 56 collates the quantity determined in S8-2 with the quantity of the medicine indicated by the prescription data. The collation process may be performed, for example, by comparing the total weight of the one or more medicines placed on the table 2 determined in S8-2 with the total weight of the medicine indicated by the prescription data.
[0098] By performing the process of FIG. 12, the user can inspect all the medicines dispensed based on the prescription by placing them on the tray 7 and placing them on the table 2.
[0099] (Example of camera image) FIG. 13(a) is a diagram showing an example of an image captured by the camera 4. The optical axis of the camera 4 is inclined relative to the vertical direction. Therefore, as in the example of FIG. 13(a), an image of the tray 7 and the medicines in the tray 7 viewed from diagonally above is captured by the camera 4. FIG. 13(b) is an example of an image captured from directly above the tray 7 by the camera 4 arranged so that the optical axis is vertical. The number of stacked PTP sheets is easier to see in the image of FIG. 13(a) than in the image of FIG. 13(b). In this way, an image in which the number of stacked PTP sheets is clear can be recorded, for example, as an image for evidence (evidence image). In this case, the number of PTP sheets can be ascertained by visually inspecting the image.
[0100] Also, by tilting the optical axis of the camera 4 with respect to the vertical direction, the marker attached to the edge of the opening of the tray 7 can be easily recognized in the image. For example, as shown in FIG. 13(a), the marker attached to the edge of the tray can be photographed from a direction in which it is easy to recognize. Therefore, the mark can be accurately recognized in the image. By using the Aruco marker as the marker, the marker can be accurately recognized even when the Aruco marker is photographed from a direction inclined with respect to the normal line of the marker's surface. The recognition accuracy of the Aruco marker is good even in an image photographed from such an oblique direction. Therefore, using the Aruco marker is advantageous for performing some kind of recognition in a photographing environment with a limited space.
[0101] (Example of a marker) Fig. 14 is a diagram showing an example of a marker. In each of Fig. 14(a) and (b), the upper diagram shows an example of the marker configuration, and the lower diagram shows an example of the marker portion of an image captured by a camera. Each of the markers M10 and M11 in Fig. 14(a) and (b) is configured by arranging white or black unit areas without gaps in two mutually orthogonal directions (vertical and horizontal). As an example, the unit areas of the markers M10 and M11 are rectangular.
[0102] In Fig. 14(a), one unit area TR of the marker M10 is indicated by a dashed line. The marker M10 in Fig. 14(a) is an example of a configuration in which two or more black unit areas are always arranged adjacent to each other. Each of the black unit areas in the marker M10 is adjacent to another black unit area. Therefore, in the marker M10, all four unit areas adjacent to one black unit area will never be white.
[0103] In the marker M11 in FIG. 14(b), the four unit areas adjacent to one black unit area KR are all white. Furthermore, in the marker M11, the eight unit areas surrounding the one black unit area KR are also all white. In this way, when there is one black unit area KR surrounded by all white unit areas, there is a high possibility that the one black unit area KR will be blurred and not recognized in the image. In particular, when the size of the marker is small, the unit area KR will be difficult to recognize in the image.
[0104] Therefore, by using a marker in which two or more black unit areas are always arranged adjacent to each other, such as marker M10 in FIG. 14(a), the recognition accuracy of the marker in the image can be improved. In other words, it is preferable to use a marker that does not include one black area surrounded by a white area, such as marker M11 in FIG. 14(b). For example, even if the size of the marker is reduced to 5 mm x 5 mm, the recognition accuracy can be maintained. By reducing the size of the marker, it is possible to prevent medicines placed on the tray from overlapping the mark. As a result, the size of the tray can be reduced.
[0105] For example, in the case of a marker that displays information in a 4-row, 4-column (4×4) white or black unit area, it is possible to assign at least numbers from 0 to 999, i.e., 1000 different numbers, to all black and white arrangement patterns. Even if patterns that include one black area surrounded by white areas are excluded from all arrangement patterns, approximately 900 patterns remain. Therefore, even if a marker that includes one black area surrounded by white areas is not used, the number indicated by the marker can be used as, for example, tray identification information.
[0106] In the process of detecting a marker in an image, the tray identification unit 51 or the target range determination unit 53 may detect the marker in a filtered image obtained by applying a filter process for enhancing edges to an image captured by a camera. This improves the recognition accuracy of the pattern (feature) of the marker.
[0107] FIG. 15 is a diagram showing an example of an image captured by the camera 4 in the operation example shown in FIG. 11. FIG. 15(a) is an example of an image when a first medicine is added to the tray 7. In this image, the area of the PTP sheet of one medicine is detected as the medicine area R1. In the medicine area R1, a barcode K1 (e.g., GS1 code) is read as medicine identification information. After the barcode K1 is read, the medicine area R1 is recorded in the memory unit 8 as a recognized area. FIG. 15(b) is an image when a second medicine is added to the tray 7. In this image, the medicine area R1, which is a recognized area, is masked. In this image, the medicine area R2 of the second medicine is detected with the area other than the medicine area R1 as the detection target range. The barcode K2 of the medicine area R2 is read. After the barcode K2 is read, the medicine area R2 is recorded as a recognized area. FIG. 15(c) is an image when a third medicine is added to the tray 7. In this image, the area other than the recognized areas R1 and R2 is set as the detection target area, and the drug area R3 of the third drug is detected, and the barcode K3 of the drug area R3 is read.
[0108] In the operation example shown in FIG. 11, the added medicine is recognized in the image captured by the camera 4. During the inspection process, if an object or person other than medicine enters the area outside the tray 7 within the shooting range of the camera 4, it may be captured in the image and erroneously detected as a medicine area. Therefore, the target range determination unit 53 can detect the markers on the tray 7 from the image captured by the camera 4 and determine the detection target range of the medicine in the image based on the detected markers. In this case, the medicine area detection unit 551 detects the medicine area within the detection target range. As a result, the medicine area is detected with the inner range of the tray 7 as the detection target range.
[0109] A plurality of markers are attached to the inner surface near the edge of the opening of the tray 7. The markers may be attached to the flange portion of the opening of the tray 7. It is preferable that the plurality of markers are attached at least to positions corresponding to the corners of the opening of the tray 7. As shown in FIG. 16, the target range determination unit 53 detects markers M1 to M9 in the image. Representative points MP1 to MP9 (MP5 for the marker M5) of each of the markers M1 to M9 are determined. For example, the target range determination unit 53 acquires the coordinates of the representative points MP1 to MP9 of each of the markers M1 to M9 in the image. In the example of FIG. 16, the marker is an Aruco marker. The Aruco marker is a square marker composed of a square black frame and a binary matrix inside the frame. The binary matrix indicates information (for example, an identifier). For example, one of the vertices of the black frame of the Aruco marker is detected as the representative point.
[0110] The area other than the hatched area in FIG. 16 shows an example of the detection target range TG1 determined based on the detected markers M1 to M9. In this example, the area inside the detected markers M1 to M9 is determined as the detection target range TG1. The target range determination unit 53 can determine, for example, an area including a polygon having vertices at the representative points MP1 to MP9 of the detected markers M1 to M9 as the detection range. In the example of FIG. 16, a polygon having vertices at the representative points MP1 to MP9 and two points on the side close to the camera 4 of the image is determined as the detection target range TG1. The medicine area detection unit 551 detects the medicine area in the detection target range TG1. That is, the area MS1 outside the detection target range TG1 is masked and excluded from the processing target. This reduces noise due to the reflection of objects or people outside the tray 7. Therefore, the accuracy of medicine detection is improved. In addition, the degree of freedom of the shape and size of the tray 7 used in the inspection work is increased.
[0111] (Example of preliminary shooting) FIG. 17 is a diagram for explaining an example of preliminary photographing and preliminary detection executed in at least one of the operation examples of FIG. 10 to FIG. 12. The control unit 57 may cause the camera 4 to photograph (preliminary photographing) an image during the user's movement of placing the tray 7 on the stand 2 or the movement of placing the medicine on the tray 7. For example, the camera 4 may be controlled to photograph a video while a movement is detected in the image of the camera 4. The medicine recognition unit 55 acquires at least one preliminary image that is preliminarily photographed. The medicine recognition unit 55 also acquires an image (mainly photographed image) photographed by the camera 4 after the tray 7 is placed on the stand 2. The images YG1 to YG3 in FIG. 17 are examples of preliminary images, and the image HG is an example of a main photographed image. In this example, the medicine recognition unit 55 executes a medicine recognition process on both the preliminary image and the main photographed image. The recognition process may be, for example, a process of reading medicine identification information (code or characters) written on the packaging material of the medicine.
[0112] In the example of FIG. 17, the medicine recognition unit 55 executes preliminary recognition of medicines for each of a plurality of auxiliary images YG1 to YG3. In the auxiliary image YG2, the code b1 of the first PTP sheet w1 is read. In the auxiliary image YG3, the codes b1 and b2 of the first and second PTP sheets w1 and w2 are read, respectively. In the actual captured image HG, the code b2 of the second sheet w2 is unreadable due to saturation. In this case, the code b2 of the second sheet w2 read in the auxiliary image YG3 is used as the recognition result.
[0113] In this way, even if the image taken is not readable, if it is readable in the preliminary image, re-imaging is not necessary. This shortens the inspection tact time and improves efficiency. For example, if the medicine identification information cannot be read in the actual image, it may be necessary to change the position or lighting conditions of the medicine and re-imaging, but if the information is readable in the preliminary image, such re-imaging is not necessary.
[0114] Preliminary photography may be performed, for example, during tray detection S1 in FIG. 12, until the tray 7 enters the photography range and is placed thereon. In the medicine recognition in S3-4 in FIG. 12, if there is a medicine area where the medicine cannot be recognized, the medicine recognition result of the same medicine area recognized in the preliminarily photographed image can be adopted. Also, in the example of FIG. 11, when the addition of a medicine to the tray is detected in S3-2, preliminary photography may be performed until the medicine enters the photography range and is placed thereon. In this case, if there is a medicine area where the medicine cannot be recognized in the medicine recognition in S3-3, the medicine recognition result of the same medicine area recognized in the preliminarily photographed image is adopted.
[0115] (Example of drug identification information reading process) Fig. 18 is a flowchart showing a specific example of the drug recognition process S3-4 in Fig. 12 or the drug recognition process S3-3 in Fig. 11. In the example of Fig. 18, first, the drug area detection unit 551 detects a drug area in an image captured by the camera 4 (S21). For example, the drug area in the image can be detected by extracting edges of the drug area by edge extraction processing. Note that the drug area detection unit 551 may detect a drug area in the detection target range by executing image processing on the detection target range determined by the above-mentioned target range determination unit 53.
[0116] S22 to S30 in FIG. 18 are processes executed for each partial image of the medicine region detected in S21. These processes are executed by the medicine recognition unit 55. The partial image of the medicine region is grayscaled (S22). Minimum bounding rectangle processing is executed for the grayscaled partial image of the medicine region (S23). In the minimum bounding rectangle processing, a minimum bounding rectangle (rectangle or square) that circumscribes the contour of the medicine region is calculated. A process is executed to rotate the partial image of the medicine region so that the sides of the minimum bounding rectangle are vertical and horizontal. That is, a process is executed to rotate the partial image according to the rotation angle of the minimum bounding rectangle that takes the rotation into account.
[0117] In S24 of Fig. 18, a filter process is performed on the partial image of the medicine region. For example, an unsharp mask and limited adaptive histogram equalization may be performed as the filter process. This makes it possible to emphasize the code or character portion in the partial image of the medicine region. The filter process (S24) improves the detection accuracy of the code or character in the partial image of the medicine region.
[0118] Code reading (S25) and character recognition (OCR) (S26) are executed in parallel on the partial image of the medicine area after the filter process of S24. That is, the OCR process is started without waiting for the result of the code reading. The code reading process is, for example, a barcode detection process. In the example of FIG. 18, if a code is detected in the code reading process (YES in S27), the medicine recognition unit 55 outputs the detected code without waiting for the result of the OCR (S30). In this case, the OCR result is discarded. If a code is not detected in the code reading process (NO in S27), the medicine recognition unit 55 waits for the result of the OCR process (S28). When the OCR result is obtained, the OCR result is output (S30).
[0119] In the process of FIG. 18, the time to wait for OCR processing when the code cannot be detected can be reduced compared to the case where OCR processing is started when the code cannot be detected by code reading.
[0120] FIG. 19 is a flow chart showing a specific example of the code reading (S25) in FIG. 18. In the example of FIG. 19, the code reading process is executed for each image obtained by rotating a partial image of the medicine area in multiple stages. In the example of FIG. 19, the process of rotating a partial image of the medicine area by one stage (S31) and the code reading process (S32) for the image rotated in S31 are repeated in multiple stages (7 times in the example of FIG. 19). The code reading process may be executed, for example, by a library that reads a barcode in an input image and returns the reading result. If a code is detected in the code reading process in any stage (YES in S33), the process exits the repeat loop and the reading result is output (S34). In the example of FIG. 19, as an example, the code reading process is executed sequentially for partial images rotated by 0, 2, -2, 4, -4, 6, and -6 degrees. This can improve the processing speed and success rate of the code reading. That is, it becomes possible to detect the code with high accuracy and high speed. 19, the rotation angle per step in S31 is 2 degrees. From the viewpoint of increasing the processing speed and accuracy, the rotation angle per step is, for example, preferably 1 to 10 degrees, and more preferably 1 to 5 degrees.
[0121] FIG. 20 is a flow chart showing a specific example of the OCR process (S26) in FIG. 18. FIG. 20 shows two processing examples, (a) and (b). In the processing of FIG. 20(a), two images are binarized in S42 for each of four images rotated by 90 degrees in S41. The two images are images in which black and white is determined for each other. As a result, eight images are generated in S41 and S42. The OCR process (S43) is performed on these eight images. The OCR process may be performed, for example, by a library that recognizes characters in an input image and returns the recognition result. From the characters recognized by the OCR process, a drug name is extracted by text matching (S44). In the text matching, for example, a recognized character string is compared with a character string of a drug name in a drug master, and a drug name that matches the character string is extracted.
[0122] In the process of FIG. 20(b), a partial region of a partial image of the medicine region is cut out (S45), and a process of estimating a rotation angle suitable for recognizing the cut-out partial region using a trained model (S46) is executed for multiple (N) different partial regions. The trained model is a model that takes an image as input and outputs a rotation angle. The rotation angle output by the trained model is one of multiple predetermined angles at intervals of a predetermined angle (e.g., 90 degrees). The trained model can be generated by machine learning (e.g., deep learning) using an image of a partial region of the image of the medicine region and a rotation angle suitable for recognizing the image of this partial region as training data.
[0123] FIG. 21 is a diagram for explaining an example of a learning process for generating a trained model. As shown in FIG. 21, in the learning process, an image of a partial region of a fixed size is cut out from an image of a medicine region and input to a model. The image of the partial region may be cut out randomly. The image of the medicine region from which the image is cut out may be preprocessed, such as by randomly changing the size, before the cutout. The model is supplied with a correct rotation angle. That is, a data set of the image of the partial region and the rotation angle becomes the teacher data. The learning process is performed by repeating a process of calculating model parameters for outputting a correct rotation angle for an input image of the partial region for multiple inputs. The model can be a neural network model.
[0124] In S47 of Fig. 20(b), the rotation angle of the partial image of the medicine region is determined based on the rotation angles of the N partial regions. The partial image of the medicine region is rotated by the determined rotation angle (S48). The rotated partial image is binarized and two images are subjected to OCR processing (S42, S43). From the characters recognized by OCR, the medicine name is extracted by text matching (S44). The processing of S42, S43, and S44 can be performed in the same manner as S42, S43, and S44 of Fig. 23(a).
[0125] FIG. 22 is a diagram for explaining an example of the process of determining the rotation angle in S45 to S47 in FIG. 20(b). In the example of FIG. 22, a rectangular partial area of a fixed size is cut out from the partial image of the medicine region. In the example of FIG. 22, five partial areas are cut out from the partial image of the medicine region. A rotation angle is estimated for each of the cut out partial areas by the trained model. The rotation angle with the largest number of rotation angles among the rotation angles of the multiple partial areas can be determined as the rotation angle of the partial image of the medicine region. In the example of FIG. 22, two sheets with different character orientations are included, and the rotation angles suitable for character recognition differ by 180 degrees between the two sheets. In this way, even if the partial image of the medicine region includes characters with different orientations, a rotation angle with a larger area is more likely to be adopted by determining the rotation angle by the majority decision process described above. This makes it possible to provide an image rotated so that more characters are oriented in a suitable direction for recognition as an image to be used for the OCR in the subsequent stage. In other words, a rotation angle suitable for recognition is more likely to be adopted. The position and number of the partial areas to be cut out may be appropriately changed according to the required accuracy and speed.
[0126] As shown in Fig. 20(b), by performing OCR processing on a partial image of the medicine area rotated at a rotation angle determined using a trained model, the number of OCR processes is reduced compared to the case of Fig. 20(a) (one-fourth in the example of S23). This allows the OCR processing time to be shortened.
[0127] (Modification of Drug Area Detection Process) The medicine area detection unit 531 of the medicine recognition unit 55 may use a trained model to determine at least the PTP sheet, pocket, PTP sheet contour, and background area in an image captured by a camera. The trained model may be a model that outputs the PTP sheet, pocket, PTP sheet contour, and background area in an input image by segmentation. The trained model can be generated by machine learning (e.g., deep learning) using as training data a data set including an image and data indicating the PTP sheet, pocket, PTP sheet contour, and background in the image.
[0128] The image captured by the camera is divided into four labels by, for example, segmentation using a trained model. As an example, each pixel of the image is labeled into one of four categories: PTP sheet, pocket, background, and PTP sheet contour by semantic segmentation. In this way, by determining the areas of the PTP sheet, pocket, and background by segmentation using a trained model, the influence of, for example, the pattern of the PTP sheet or background and the lighting environment on the recognition accuracy is reduced. Therefore, when there are various patterns on the PTP sheet or background, or in various lighting environments, these areas can be determined with high accuracy. In addition, by classifying the areas of the PTP sheet contour in addition to the PTP sheet, pocket, and background by segmentation using a trained model, the contour of the overlapping PTP sheets can be detected with high accuracy. As a result, for example, the number of PTP sheets can be determined based on the image captured by the camera 4.
[0129] (Example of PTP sheet number detection process) The medicine recognition unit 55 detects the contour of the PTP sheet in the image captured by the camera 4, and can determine the number of PTP sheets based on the number of intersections between the reference line and the detected contour of the PTP sheet. The contour of the PTP sheet can be detected, for example, by edge detection or segmentation using a trained model.
[0130] The medicine recognition unit 55 acquires a partial image of the PTP sheet area including the contour of the PTP sheet recognized in the image. The medicine recognition unit 55 obtains a minimum circumscribing rectangle of the PTP sheet area and rotates the partial image to fit the rectangle. The number of contour lines of the PTP sheet that pass through the rectangle and intersect with each of two straight lines that are perpendicular to each other is counted. The number of PTP sheets is determined based on the number of contour lines of the PTP sheet on each line.
[0131] FIG. 23 is a diagram showing an example of the number of intersections between the contour line of a PTP sheet and a straight line. In FIG. 23(a), the straight line L1 crossing the overlapping PTP sheets and the contour line of the PTP sheet intersect at three points. Since the contour line of the top PTP sheet intersects at two points with the straight line L1, when the intersections are three points, the number of PTP sheets can be determined as 3-1=2. Also, as shown in FIG. 23(b), the number of PTP sheets may be determined based on the number of intersections between the contour line and each of two or more parallel straight lines L1 to L3, respectively. This allows the number of PTP sheets to be determined with high accuracy even when the contour line is blurred or noise occurs in the image. As shown in FIG. 23(c), the number of PTP sheets can be determined based on the number of intersections between the contour line and each of a plurality of straight lines L4 to L6 crossing the PTP sheet. This allows for higher accuracy. In the case of FIG. 23, the number of intersections between the transverse straight lines L1 to L3 in FIG. 23(b) and the contour line is 2, 3, and 3. The corresponding number of PTP sheets is 1, 2, and 2. According to the majority rule, the number of PTP sheets can be determined to be 2. The number of intersections between the longitudinal straight lines L4 to L6 in FIG. 23(c) and the contour line is 2, 3, and 3, and the corresponding number of PTP sheets is 1, 2, and 1. The number of PTP sheets can be determined to be 1 by majority rule. In this way, when the number of sheets determined differs depending on the direction of the reference straight line, the largest number among the different numbers can be determined as the final number of PTP sheets (2 in the example of FIG. 23).
[0132] (Example of fractional PTP sheet detection) The medicine recognition unit 55 may detect the size of the topmost fractional sheet of the overlapping PTP sheets in the image captured by the camera 4. The number of tablets on the fractional sheet can be determined based on the size of the fractional sheet. The detection process of the fractional sheet may include, for example, a process of detecting the most focused edge in the image captured using the autofocus (automatic focus) of the camera 4. The edge detected in this manner is likely to be the edge of the topmost PTP sheet of the overlapping PTP sheets. For example, the most focused edge can be extracted by extracting an edge with a higher contrast than other edges among the edges detected in the image. This allows the edge of the topmost PTP sheet of the overlapping PTP sheets to be detected. The medicine recognition unit 55 can detect the area surrounded by the edges of this topmost PTP sheet as the area of the fractional sheet.
[0133] As an example, a case where a fraction sheet is placed on top of an integer sheet will be described. The contrast of the edge E1 of the fraction PTP sheet P1 placed on top is higher than the contrast of the edge E2 of the integer PTP sheet P2 in the lower layer. Based on this difference in contrast, the edge of the top PTP sheet is extracted as the edge of the fraction sheet. For example, an area surrounded by edges whose extracted contrast is higher than other edges is extracted. In this example, multiple areas are extracted. Of these multiple areas, an area (area HR in FIG. 29(d)) whose shape and area at least meet a predetermined condition is determined to be the area of the fraction sheet. The number of tablets contained in the fraction sheet can be determined based on the area of the area HR of the fraction sheet.
[0134] The medicine inspection system 10 may display on the display 5b the medicine names of the medicines in the image recognized by the medicine recognition unit 55, superimposed on the image. In this case, the names of the medicines recognized in each medicine region can be displayed in an area overlapping each medicine region in the image. In this case, a medicine name frame for displaying the medicine name for each medicine region in the image is determined by processing of the computer 5a. The medicine name frame is determined, for example, at a position (predetermined position) having a predetermined positional relationship with each medicine region. As an example, the size of the medicine name frame surrounding the displayed medicine name is determined based on the number of characters of the medicine name. The position of the medicine name frame in the image is determined so that the center of the medicine region in the image coincides with the center of the medicine name frame.
[0135] In an image, when multiple medicine regions are located close to each other, multiple medicine names are displayed overlapping. In this case, for example, as shown in Fig. 24, if medicine name frames N1 to N6 are placed at predetermined positions for each of multiple medicine regions R1 to R6, the medicine names may be displayed overlapping, resulting in poor visibility. Therefore, the medicine inspection system 10 determines the position of the medicine name frame for the medicine region to a position moved from the predetermined position, thereby preventing multiple medicine names from overlapping.
[0136] As an example, the medicine inspection system 10 judges whether or not a plurality of medicine name frames overlap when the medicine name frames are placed at a predetermined position for each of a plurality of medicine regions. When a plurality of medicine name frames overlap, a destination position of the medicine name frame is determined based on the coordinates of a representative point (e.g., the upper left point) of the overlapping medicine name frames so that the plurality of medicine name frames do not overlap. In this case, the destination position may be determined so that the amount of movement is minimized. Also, the predetermined position of the medicine name frame with respect to the medicine region may be determined so that the longitudinal direction of the medicine region and the longitudinal direction of the medicine name frame coincide with each other.
[0137] FIG. 25 is a diagram showing an example of a movement process of a medicine name frame. In the example of FIG. 25, of the overlapping medicine name frames N1 and N2, the medicine name frame N1 whose representative point (upper left end point in the example of FIG. 25) is closest to the upper left of the image is fixed, and the other medicine name frame N2 is moved. A plurality of virtual regions A1 to A4 are defined based on the center of the fixed medicine name frame N1, and the movement direction is determined depending on which region the representative point (upper left end point) of the medicine name frame N2 to be moved belongs to. The movement amount is determined according to the overlap amount. For example, the movement amount is determined according to the length in the movement direction of the overlapping regions of the medicine name frames N1 and N2. In the example of FIG. 25, the medicine name frame N1 is divided into four regions, a first region (first quadrant) A1 on the upper right, a second region (second quadrant) A2 on the upper left, a third region (third quadrant) A3 on the lower left, and a fourth region (fourth quadrant) A4 on the lower right, with respect to the center of the medicine name frame N1.
[0138] As shown in FIG. 25(a), when the representative point of the medicine name frame N2 is in the first quadrant A1, it is moved to the right (positive direction of the x-axis) by the length of the overlapping medicine name frame N2 in the x-direction. As shown in FIG. 25(b) and (c), when the representative point of the medicine name frame N2 is in the second quadrant A2 or the third quadrant A3, it is moved downward (negative direction of the y-axis) by the length of the overlapping medicine name frame N2 in the y-direction. As shown in FIG. 25(d), when the representative point of the medicine name frame N2 is in the fourth quadrant A4, it is moved in the short direction of the overlapping area of the medicine name frames N1 and N2 by the length of the short direction of the overlapping medicine name frame N2. In the example of FIG. 25(d), the overlapping area of the medicine name frames N1 and N2 is shorter in the y direction than in the x direction. Therefore, the medicine name frame N2 is moved in the y direction by the length of the overlapping area in the y direction. In the example of FIG. 25, the movement amount of the medicine name frame can be minimized.
[0139] When there are multiple sets of overlapping drug name frames in an image, the drug name frames can be sorted in order of the coordinates of the representative points of the drug name frames being closest to the top left point of the image, and the sorted drug name frames can be fixed in order, and other drug name frames that overlap with the fixed drug name frames can be moved. This simplifies the movement process.
[0140] FIG. 26 is a diagram showing an example of the state before and after the movement of the medicine name frame in the image. FIG. 26(a) is an example in which the predetermined positions of the medicine name frames N1, N21, and N31 for the medicine regions R1, R2, and R3 are constant regardless of the longitudinal direction of the medicine region. FIG. 26(b) is an example in which the predetermined positions of the medicine name frames N1, N21, and N31 for the medicine regions R1, R2, and R3 are determined so that the longitudinal direction coincides with the medicine region. In the example of FIG. 26(b), the movement amount from the predetermined position of the medicine name frames N1, N22, and N32 is shorter than that in the example of FIG. 26(a). In this way, by arranging the medicine name frames so that the longitudinal direction coincides with the medicine region, the movement amount for eliminating overlap can be shortened. In this case, when determining the predetermined position of the medicine name frame for each medicine region, the vertical and horizontal lengths of the medicine region are compared. A predetermined position of the medicine name frame relative to the medicine area is determined so that if the medicine area is long vertically, the medicine name frame is written vertically, and if the medicine area is long horizontally, the medicine name frame is written horizontally.
[0141] The medicine inspection system 10 may have a function of linking with a portable inspection system. FIG. 27 is a functional block diagram showing an example configuration of such a medicine inspection system 10. The medicine inspection system 10 of FIG. 27 includes a weighing device 1, a platform 2, a tray identification unit 51, a weight calculation unit 52, and a collation unit 56, as well as a linking unit 58, which communicates data with the portable inspection system 20. The medicine inspection system 10 is capable of communicating with the portable inspection system 20. The medicine inspection system 10 and the portable inspection system 20 are capable of communicating with a higher-level system 30, and can obtain prescription data from the higher-level system 30. The higher-level system 30 may be, for example, a prescription computer system.
[0142] The portable inspection system 20 may be, for example, a picking assistance system that assists in picking medicines. The portable inspection system 20 can be implemented by a mobile terminal such as a smartphone or a PDA. The portable inspection system 20 has a reader 21, and inspects medicines by collating medicine information read by the reader 21 with prescription data. The computer included in the portable inspection system 20 has a tray identification unit 22 that identifies a tray based on a marker attached to the tray read by the reader 21, a medicine information acquisition unit 23 that acquires medicine information based on the medicine code read by the reader 21, a collation unit 24 that collates the medicine information with prescription data, and a linking unit 26 that performs data communication with the medicine inspection system 10. The linking unit 26 transmits to the medicine inspection system 10, in addition to the medicine information read by the reader 21, tray information (e.g., a tray identifier (tray ID)) and prescription information (e.g., a prescription ID) corresponding to the medicine information.
[0143] The linking unit 58 of the medicine inspection system 10 receives these pieces of information from the portable inspection system 20. The weight calculation unit 52 calculates the weight of the medicine on the tray. If the tray indicated by the tray information received from the portable inspection system 20 is the same as the tray detected by the tray identification unit 51, i.e., the tray placed on the platform 2, the weight calculation unit 52 can calculate the weight of the medicine based on the medicine information received from the portable inspection system 20. In addition, the collation unit 56 collates the weight of the medicine with the prescription data based on the medicine information and prescription information received from the portable inspection system 20. This makes it possible to inspect whether the weight of the medicine matches the prescription. The collation unit 58 may transmit the inspection result of the weight of the medicine to the portable inspection system 20.
[0144] In this way, the portable inspection system 20 inspects the picked medicines, and the user places the tray containing the medicines on the platform 2 of the weighing device 1 of the medicine inspection system 10, allowing the medicine inspection system 10 to inspect whether the weight of the medicines matches the prescription. In this way, the markers on the tray are read by both the portable inspection system 20 and the medicine inspection system 10, respectively, to identify the tray, allowing the two systems to inspect the medicines placed on the tray in conjunction with each other. The markers on the tray may be, for example, a GS1 code, a QR code (registered trademark), or an AR code, etc.
[0145] In drug recognition during the inspection process of the drug inspection system 10, drug master data is referenced. The drug master data is stored, for example, in the storage unit 8 accessible to a computer. In the inspection process, if there is no master data for a drug that corresponds to the drug master data accessible to the computer, a no-master error occurs. When a no-master error occurs, the computer can automatically generate a request for the required master data and request it from the supplier of the master data, or record it in the storage unit 8. The recorded request is transmitted to the supplier under specified conditions. This makes it possible to efficiently obtain master data for the required amount of drugs. For example, the amount of communication can be reduced compared to the case where all masters or differential masters are downloaded from the supplier all at once.
[0146] Medicine inspection system 10 may have a function of registering the weight of a medicine measured using weighing device 1 in the medicine master data. For example, if the weight of a medicine is not registered in the medicine master data, medicine inspection system 10 may register the weight of the medicine measured using weighing device 1 in the medicine master data in response to a user's instruction. In this case, medicine inspection system 10 can measure the weight of the medicine visually inspected by the user using weighing device 1, calculate the weight per unit amount of the medicine, and register it in the medicine master data. This allows the user to efficiently register the weight of the medicine being inspected in the medicine master data with minimal work.
[0147] As an example, the medicine inspection system 10 executes the following processes in order as the inspection process: determining the prescription to be inspected, determining the medicine to be inspected, and receiving the input of the visual inspection result of the medicine amount by the user. The process of determining the prescription to be inspected may be, for example, a process of receiving the selection or designation of the prescription to be inspected from the user on the screen. The process of determining the medicine to be inspected may be at least one of receiving the selection of the medicine from the user on the screen, acquiring the medicine information based on the information read by the user with the reader 6, or acquiring the medicine information from the image captured by the camera 4. The input reception of the visual inspection result may be a process of receiving the input of the result of the user visually checking whether or not the amount of the medicine to be inspected displayed on the screen is as prescribed. When the user inputs the visual inspection result that the amount of the medicine is as prescribed, the medicine inspection system 10 outputs an instruction to the user to measure the visually inspected medicine. The measurement instruction may be, for example, an instruction to place the medicine on the platform 2 of the weighing device 1. A weighing instruction for a visually inspected medicine may be executed, for example, when the weight of the medicine to be inspected is not registered in the medicine master data. When the user places the medicine on platform 2, weight calculation unit 52 calculates the weight per unit amount of the medicine based on the measurement value of weighing device 1. The calculated weight per unit amount of the medicine is registered in the medicine master data.
[0148] As an example, the medicine inspection system 10 sequentially executes the processes of determining the prescription to be inspected, determining the medicine to be inspected, and judging whether the medicine is as prescribed using the inspection method selected by the user. In this case, the user may be able to select the inspection method for the medicine before determining the medicine to be inspected. For example, on a screen where the user selects the medicine to be inspected, a user interface tool (e.g., a button, etc.) for inputting the inspection method may be displayed corresponding to each medicine. In this way, the inspection method is determined before determining the medicine to be inspected, so that it is possible to avoid presenting unnecessary information to the user depending on the inspection method.
[0149] The inspection method selectable by the user may be at least two of, for example, weight inspection, which judges whether the weight of the medicine measured using the weighing device 1 is as prescribed, visual inspection, which accepts an input of a visual inspection result in which the user visually confirms whether the medicine is as prescribed, and code check, which accepts an input of medicine information read by the reader 6. For example, in the case of weight inspection, weight information of the medicine to be inspected needs to be registered in the medicine master data. When the user selects visual inspection as the inspection method for a certain medicine, even if weight information is not registered in the medicine master data for that medicine, there is no need to check this fact, and no need to present a message to that effect to the user. As described above, by selecting the inspection method before deciding the medicine to be inspected, these unnecessary processes can be omitted.
[0150] The medicine inspection system 10 performs the following inspection process in order: determining the prescription to be inspected, determining the medicine to be inspected, acquiring information on the medicine prepared by the user, judging whether the medicine prepared by the user is as prescribed or not, and outputting the judgment result. During the inspection process, that is, between the time when the prescription to be inspected is determined and the time when the judgment result is output, the prescription may be corrected in the upper system that supplies the prescription data. In this case, the prescription correction information is notified to the medicine inspection system 10. When the medicine inspection system 10 receives prescription correction information during the inspection process, it can notify the user that the prescription has been corrected. Together with this notification, the medicine inspection system 10 may receive at least one of the following instructions from the user: canceling the currently executing inspection process, and executing the inspection process with the corrected prescription data. This allows the inspection to be redone with the corrected prescription when a prescription correction is made during the inspection.
[0151] The medicine inspection system 10 may acquire weighing results from another weighing system used to dispense medicines of the same prescription data, based on prescription data designated by the user as an inspection target. For example, in a weighing system that automatically weighs and dispenses powdered medicines based on prescription data, weighing result data is recorded in association with prescription data (e.g., prescription ID). The medicine inspection system 10 can acquire and display this weighing result data based on an instruction from a user. For example, the user causes the medicine inspection system 10 to read a QR code (registered trademark) of the prescription ID, and inputs an instruction to acquire the weighing result data. The medicine inspection system 10 can accept this prescription ID and the instruction input, acquire the weighing result data of the other system associated with the prescription ID, and output (e.g., display, print, etc.) to the user.
[0152] The medicine inspection system 10 may have a function of recognizing and weighing medicines and recording the results together with images for purposes other than inspection processing. For example, a pharmacy may distribute some of its stocked medicines to another pharmacy (transfer / acquisition of medicines between pharmacies). In this case, it may be necessary to record the names and quantities of the medicines prepared for distribution as evidence. Even in such a case, the medicine inspection system 10 can record the quantity based on the weight of the medicine to be distributed placed on the platform 2 attached to the weighing device 1, the medicine identification information read by the reader 6 or camera 4, and the image of the medicine photographed by the camera 4 in the memory unit 8 for evidence, and output (for example, print) them.
[0153] The above-described embodiment is an example of the present invention, and the present invention is not limited to this example. Furthermore, the configurations and processing functions described in the above-described embodiment can be selected and combined as desired. [Explanation of symbols]
[0154] 1: personal record unit, 2: image recording unit, 3: recognition result acquisition unit, 4: group determination unit, 5: duplicate candidate display unit, 10: medicine recognition support system, 20: recognition system
Claims
1. a weighing device for measuring the weight of an object to be measured; a base attached to a load receiving portion of the weighing device; A windshield member; a camera that includes the platform attached to the load receiving portion of the weighing device in its imaging range, A medicine inspection system, wherein the windshield member is not in contact with the load receiving portion and is positioned outside the load receiving portion in a position that blocks wind from reaching the underside of the table.
2. The medicine inspection system according to claim 1, A drug inspection system, wherein the optical axis of the camera is arranged at an angle relative to the vertical direction.
3. The medicine inspection system according to claim 1 or 2, A medicine inspection system, wherein the windshield member is spaced apart from the base and the load receiving portion and extends from below the base to the side of the base.
4. The medicine inspection system according to claim 1 or 2, a tray identification unit that detects a marker attached to a tray placed on the table from the image captured by the camera and identifies the tray based on the detected marker; The medicine inspection system further comprises a weight calculation unit that calculates the weight of the medicine on the tray placed on the platform using the identification result of the tray identification unit and the measurement value by the weighing device.
5. The medicine inspection system according to claim 1 or 2, a target range determination unit that detects markers attached to a tray placed on the table from the image captured by the camera and determines a detection target range of medicines in the image based on the detected markers; The medicine inspection system further includes a medicine area detection unit that detects a medicine area in the detection target range determined by the target range determination unit.
6. The medicine inspection system according to claim 4, The marker is configured by white or black unit areas arranged without gaps in two mutually orthogonal directions (for example, vertically and horizontally), A medicine inspection system in which two or more black unit areas are always arranged adjacent to each other in the marker.
7. The medicine inspection system according to claim 1 or 2, an added medicine detection unit that detects the addition of a medicine to the platform by at least one of an image captured by the camera and a measurement by the meter when a medicine is added to the platform; a weight calculation unit that calculates the weight of the drug added to the platform based on the increase in weight measured by the weighing scale; The medicine inspection system further comprises a collating unit that collates the quantity based on the weight of the added medicine with prescription data.
8. 1. A drug inspection program comprising: A computer connected to a camera that captures the scale and the platform attached to the load receiving part of the scale, A process of detecting a marker on a tray placed on the table from an image captured by the camera; identifying the tray based on the detected marker; and calculating the weight of the medicine on the tray placed on the platform using the tray identification result and the weight value measured by the weighing scale.
9. 1. A drug inspection program comprising: A computer connected to a camera that captures the scale and the platform attached to the load receiving part of the scale, A process of detecting a marker attached to a tray placed on the table from an image captured by the camera; determining a detection target range of the drug in the image based on the detected marker; and detecting a medicine area in the determined detection target range.
Citation Information
Patent Citations
Inspection support device and inspection support method
JP2016047223A
