A dispensing method and a dispensing vehicle for traditional Chinese medicine decoction pieces

Abstract

This invention discloses a method for dispensing traditional Chinese medicine (TCM) decoction pieces and a dispensing cart. The dispensing method includes the following steps: creating a "doupu" (a system of measuring the spatial coordinates of each TCM decoction piece in a medicine cabinet); retrieving the target decoction piece according to the prescription and acquiring a real-time image of the target decoction piece; based on the current "doupu" coordinates of the target decoction piece, calling a sub-model of the corresponding target decoction piece in an image recognition model to recognize and verify the real-time image; wherein, the sub-model includes images of the target decoction piece and its neighboring decoction pieces in the "doupu"; determining whether there is an anomaly in the current retrieval and issuing a warning message when an anomaly is detected. This invention effectively solves the technical problem of easy misidentification of decoction pieces with similar appearances by establishing a "doupu" to coordinate the position of the decoction pieces and calling a sub-model specifically trained for the target decoction piece and its fixed neighboring decoction pieces based on the coordinates for verification, significantly improving the recognition accuracy and reliability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent technology of traditional Chinese medicine, specifically, it relates to a method for dispensing Chinese herbal medicine slices and a dispensing cart. Background Technology

[0002] When dispensing bulk medicinal herbs according to prescriptions in traditional Chinese medicine pharmacies, the process relies heavily on the experience and focus of the dispensing pharmacist. In the traditional process, the pharmacist must refer to the prescription and pick out and weigh various medicinal herbs one by one from numerous medicine drawers. This process is labor-intensive and highly repetitive, and is prone to problems such as incorrect picking, omissions, inaccurate dosages, or contamination due to fatigue or negligence, which directly affect the safety and efficacy of medication.

[0003] To improve the accuracy and efficiency of dispensing, existing technologies have attempted to introduce intelligent equipment and proposed using image recognition technology to assist in the identification of medicinal herbs. However, there are many types of Chinese medicinal herbs, often numbering in the thousands, and many of them are extremely similar in appearance and color, posing a challenge to easily confuse similar herbs. This makes it difficult to meet the needs of accurate verification in actual dispensing scenarios. Therefore, there is an urgent need for an intelligent dispensing method that can effectively improve the accuracy of verification.

[0004] In view of this, the present invention is proposed. Summary of the Invention

[0005] In order to solve at least some of the above-mentioned problems, the first objective of this invention is to provide a method for dispensing Chinese herbal medicine slices, which can improve the verification accuracy during dispensing. The second objective of this invention is to provide a dispensing cart for Chinese herbal medicine slices.

[0006] To solve the above-mentioned technical problems, a first aspect of the present invention provides a method for dispensing traditional Chinese medicine decoction pieces, comprising the following steps: A grid for preparing medicinal slices, which records the spatial coordinates of each medicinal slice in the medicine cabinet; The target medicinal slices are captured according to the prescription, and real-time images of the target medicinal slices are collected. Based on the current target medicinal slice's coordinates in the image recognition model, the corresponding sub-model of the target medicinal slice is invoked to recognize and verify the real-time image; wherein, the sub-model includes images of the target medicinal slice and its neighboring medicinal slices in the image recognition model; Determine if there are any anomalies in the current data capture, and issue an alert when an anomaly is detected.

[0007] In some embodiments, the image recognition model includes multiple sub-models for different target medicinal slices, and the step of training the sub-models includes: Collect sample images of the target medicinal slice and its neighboring medicinal slices in the distillation table; The sample images are labeled to form a training set; Associating the decoction pieces in the training set with their spatial coordinates in the prescription drawer arrangement, and training to obtain a sub-model corresponding to the target decoction piece.

[0008] In some embodiments, the labeling processing on the sample images comprises: cropping the sample images according to preset grid regions to form a plurality of regional images, wherein different regional images respectively correspond to different decoction pieces; selecting a plurality of regional images containing the target decoction piece, and labeling the types of decoction pieces contained in each regional image.

[0009] In some embodiments, performing recognition and verification on the real-time image comprises: performing comparative analysis on the real-time image and the sub-model corresponding to the target decoction piece; if the decoction piece in the real-time image does not correspond to the sub-model, determining that an abnormality exists in the current grabbing operation; Preferably, the real-time image comprises an image of the target decoction piece and decoction pieces adjacent to the target decoction piece in the prescription drawer arrangement, and the recognition and verification on the real-time image further comprises: performing multi-region search on the real-time image to obtain window images corresponding to different decoction pieces; recognizing the types of decoction pieces of the window images and the distribution positions thereof in the real-time image, and performing comparative analysis with the sub-model; if the types of decoction pieces of the window images and the distribution positions thereof in the real-time image do not correspond to those of the sub-model, determining that an abnormality exists in the current grabbing operation.

[0010] In some embodiments, the spatial coordinates of the prescription drawer arrangement comprise horizontal coordinates and vertical coordinates, and the real-time image comprises a horizontal image and a vertical image; when the recognition and verification is performed on the real-time image, the horizontal image and the vertical image are subjected to comparative analysis at the same time.

[0011] Further, the grabbing the target decoction piece according to the prescription comprises: planning a dispensing path and a grabbing sequence according to the prescription and the prescription drawer arrangement, and performing grabbing in sequence.

[0012] In some embodiments, the method further comprises: if the early warning is confirmed to be a false alarm, adjusting the adjacent layout of related decoction pieces in the prescription drawer arrangement, and / or optimizing the corresponding sub-model.

[0013] In a second aspect of the present invention, there is provided a traditional Chinese medicine decoction piece dispensing cart for implementing the above traditional Chinese medicine decoction piece dispensing method, comprising: a cart body; an image acquisition device arranged on the cart body and configured to acquire a real-time image of a target decoction piece; The control unit, which is mounted on or communicatively connected to the vehicle body, is configured to: Store or access the herbal medicine recipe catalog; Determine the target medicinal slices based on the prescription and the biographies; The sub-model corresponding to the current target medicinal slice is invoked to identify and verify the real-time image; Based on the verification results, the control and early warning device will issue anomaly warnings.

[0014] In some implementations, it also includes: A weighing module, installed on the vehicle body, is used to weigh the grabbed beverage slices; The control unit is also configured to compare the weighing result with the prescription dosage and to issue an early warning when the dosage deviation exceeds the limit.

[0015] Furthermore, it also includes a human-computer interaction terminal, which is installed on the vehicle body and is used to display prescription information, dispensing guidance information and warning information.

[0016] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art.

[0017] This invention establishes a coordinate system for identifying the location of medicinal slices by establishing a grid system. Based on these coordinates, it calls sub-models specifically trained for the target medicinal slice and its fixed neighboring medicinal slices for verification. This transforms the general problem of identifying thousands of medicinal slices into a large number of small-scale, environmentally fixed target verification problems. Since each sub-model needs to distinguish far fewer objects (the target medicinal slice and a few fixed neighbors) than all medicinal slice types, and the background environment is relatively fixed, the learning difficulty and confusion of the model are greatly reduced. As a result, in the actual complex dispensing scenarios, it achieves accuracy and reliability far exceeding that of general recognition models, effectively solving the technical problem of easy misidentification of similar-looking medicinal slices and significantly improving recognition accuracy and reliability.

[0018] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, as part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation of the invention. Obviously, the drawings described below are merely some embodiments, and those skilled in the art can obtain other drawings based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of the dispensing method for traditional Chinese medicine decoction pieces provided in the embodiments of this application; Figure 2 This is another flowchart of the method for dispensing traditional Chinese medicine decoction pieces provided in the embodiments of this application; Figure 3 This is another flowchart of the method for dispensing traditional Chinese medicine decoction pieces provided in the embodiments of this application; Figure 4 This is a system structure diagram of the dispensing cart for traditional Chinese medicine decoction pieces provided in the embodiments of this application.

[0020] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0022] In the description of this invention, it should be noted that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0023] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] See Figure 1 and Figure 2 In a first aspect, the present invention provides a method for dispensing traditional Chinese medicine decoction pieces, the method comprising the following steps: Step 1: Create a catalog of medicinal slices, which records the spatial coordinates of each medicinal slice in the medicine cabinet; Step 2: Extract the target medicinal slices according to the prescription and acquire real-time images including the target medicinal slices; Step 3: Based on the current target slice's coordinates in the image recognition model, call the corresponding sub-model of the target slice to recognize and verify the real-time image; wherein, the sub-model includes images of the target slice and its neighboring slices in the image recognition model; Step 4: Determine if there are any anomalies in the current crawling process, and issue an alert when an anomaly is detected.

[0025] Specifically, when creating the medicine cabinet's catalog, the medicine cabinet is systematically managed, and a catalog of medicine slices is established. The catalog records the three-dimensional spatial coordinates of each medicine slice in the medicine cabinet shelf or medicine drawer. For example, it can include the horizontal row and column number and the vertical layer height information. In other words, the spatial coordinates are based on the three-dimensional space of the medicine cabinet to establish row, column, and layer numbers. For example, a medicine slice can be recorded as located in the 3rd row, 5th column, and 2nd layer of area A, providing a spatial reference for subsequent intelligent identification and scheduling.

[0026] When a pharmacist or operator performs a dispensing task based on an electronic or paper prescription, the system indicates the target herbal medicine to be picked up based on the prescription information. The operator then moves to the medicine cabinet location corresponding to the coordinates of the target herbal medicine and performs the picking action.

[0027] During this process or after the grabbing action is completed, real-time images are captured using image acquisition devices such as high-definition cameras.

[0028] Subsequently, the control unit queries the Doupu database to obtain the spatial coordinates of the target medicinal slice according to the target medicinal slice indicated by the current prescription. Based on these coordinates, the system calls a sub-model from a pre-trained image recognition model library to verify the target medicinal slice. The training data of this sub-model not only includes the image features of the target medicinal slice itself, but also the image features of several neighboring medicinal slices recorded in the Doupu. Specifically, the sample images used by the sub-model during training include the target medicinal slice and several fixed medicinal slices around it determined according to the Doupu.

[0029] The system calls the sub-model to analyze the acquired real-time images. The model will output the recognition results, such as determining whether the central subject of the image is the expected target medicinal slice (i.e., the target medicinal slice in the real-time image). If so, it is determined that there is no abnormality in the capture. It also determines whether the target medicinal slice in the real-time image matches the neighboring medicinal slices in the sub-model. If so, it is determined that there is an abnormality in the capture, indicating that the pharmacist has captured the medicinal slices that are next to the target medicinal slice.

[0030] Once an anomaly is detected, the system will issue an alarm via the sound and light alarm on the dispensing vehicle or display a prominent warning message on the human-machine interface terminal, forcibly interrupting the current operation process and reminding the pharmacist to review and correct the error, thereby preventing the error from flowing into the next stage as soon as possible.

[0031] This invention establishes a coordinate system for identifying the location of medicinal slices by establishing a grid system. Based on these coordinates, it calls sub-models specifically trained for the target medicinal slice and its fixed neighboring medicinal slices for verification. This transforms the general problem of identifying thousands of medicinal slices into a large number of small-scale, environmentally fixed target verification problems. Since each sub-model needs to distinguish far fewer objects (the target medicinal slice and a few fixed neighbors) than all medicinal slice types, and the background environment is relatively fixed, the learning difficulty and confusion of the model are greatly reduced. As a result, in the actual complex dispensing scenarios, it achieves accuracy and reliability far exceeding that of general recognition models, effectively solving the technical problem of easy misidentification of similar-looking medicinal slices and significantly improving recognition accuracy and reliability.

[0032] In some implementations, the image recognition model is not a single, large and complex recognition model that encompasses all Chinese herbal medicine pieces. Instead, it is a modular model library composed of a large number of sub-models, each of which corresponds to and serves a specific target herbal medicine piece recorded in the medicine cabinet catalog and its neighboring herbal medicine pieces.

[0033] See Figures 1 to 3 The steps for training the sub-model include: First, sample images are acquired. The focus of the acquisition is not only on the image of the target medicinal slice itself, but also on the combined images of the target slice and its neighboring slices, based on their spatial coordinates in the actual medicine cabinet. For example, if the target medicinal slice A is recorded as having slice B as its left neighbor and slice C as its right neighbor in the cabinet, then the sample image should include a scene where A, B, and C are in the same frame, simulating the real scene that the camera might capture during actual operation. This ensures that the model learns the visual features of the target medicinal slice in its specific spatial context.

[0034] The collected sample images are then labeled to form a training set. During labeling, it is necessary to clearly indicate which type of medicinal slice corresponds to which region in the image. For example, in an image containing medicinal slices A, B, and C, it is necessary to accurately label which pixel regions belong to A, which belong to B, and which belong to C, and assign the correct category label to each region. This process transforms the raw image data into a machine-understandable set of image-labels.

[0035] Then, spatial coordinate association is performed. The system associates the labeled category information of the medicinal slices in the training set with the spatial coordinates of the medicinal slice in the Doupu database. This means that during the model training process, the algorithm not only learns to recognize the appearance of medicinal slice A, but also knows that under standard conditions, medicinal slices B and C should appear at specific locations in the image of medicinal slice A (corresponding to its physical neighbors). Through this association, the trained sub-model inherently encodes the geographical location and visual background information of the target medicinal slice.

[0036] Finally, deep learning algorithms, such as convolutional neural networks, are used to learn from the above-mentioned associated training set, optimize the model parameters, and obtain a sub-model for verifying the specific target herbal medicine slice and its fixed neighbor environment. This sub-model is then stored in the model library for use during dispensing verification.

[0037] This implementation method decomposes the massive and complex problem of identifying all medicinal slices into hundreds of small-scale sub-problems that verify specific medicinal slices and their few neighbors, greatly reducing the learning difficulty of each sub-model. The model only needs to distinguish a few fixed medicinal slices and the background, avoiding confusion caused by subtle differences among thousands of medicinal slices, thereby achieving extremely high recognition accuracy in engineering practice.

[0038] Furthermore, the amount of training data required for a single sub-model is far less than that of a general large model, significantly reducing the cost of data collection and annotation. When the medicine cabinet's layout is adjusted (such as changing the position of medicinal slices) or when new medicinal slices need to be added, it is only necessary to re-collect the combined image of the changed medicinal slices with their new neighbors, and train or update the corresponding individual sub-models. There is no need to retrain the entire large system, resulting in low maintenance and iteration costs and fast response speed.

[0039] In some implementations, see Figures 1 to 3 To further refine the sample image annotation process, a structured pre-defined grid cropping annotation method is adopted: the sample image is cropped according to the pre-defined grid area to form multiple region images, and different region images correspond to different medicinal slices; several region images including the target medicinal slices are selected, and the types of medicinal slices contained in each region image are labeled.

[0040] Specifically, in the annotation system or annotation tool, an image region division template is predefined and fixed. For example, the template is a five-grid pattern, which logically divides any input sample image into five fixed rectangular regions: the upper left region, the upper right region, the lower left region, the lower right region, and the center region. The coordinate parameters of the template are preset and are independent of the specific image content, ensuring consistency in the processing of all samples.

[0041] When annotating the sample image, the system first automatically crops an original sample image containing various medicinal slices into five independent region images based on a preset grid template. These five small images correspond to different parts of the original image.

[0042] Next, the annotators or automatic filtering program select several images from these five regional images that actually contain valid medicinal slice information, i.e., select N from five, where N is less than 5. For example, the image in the central region is likely to contain the target medicinal slice for this training, while the image in a certain corner region may just contain its neighboring medicinal slice.

[0043] For each selected area image, the annotator determines which type of medicinal slice is contained in the small image and assigns it the corresponding category label. For example, the central image is labeled as Astragalus membranaceus, and the upper left image is labeled as Angelica sinensis.

[0044] The pre-defined grid template ensures the standardization of the cropping area, reducing subjective bias and noise introduced by different annotators' different understandings of the target range. All training data are generated based on unified region division rules, with a regular format, which helps the model learn features more stably and efficiently, and improves the consistency and reliability of the final trained sub-models.

[0045] Meanwhile, the complex multi-target annotation of the whole image is broken down into simple classification of several small images, which greatly reduces the operational difficulty and attention requirements of the annotators. Annotators do not need to draw the outline precisely, but only need to make quick category judgments, which makes the annotation speed increase exponentially and significantly reduces the time and manpower required to build a large training set.

[0046] In some implementations, see Figure 1 and Figure 2 The process of recognizing and verifying real-time images includes the following steps: The real-time image is compared and analyzed with the sub-model corresponding to the target medicinal slice; If the slices in the real-time image do not correspond to the sub-model, it is determined that there is an anomaly in the current capture operation. Specifically, the system calls the sub-model to analyze the acquired real-time images, and the model will output the recognition results. For example, it determines whether the central subject of the sub-model image is the expected target medicinal slice (i.e., the target medicinal slice in the real-time image). If so, it determines that there is no abnormality in the capture. It also determines whether the target medicinal slice in the real-time image matches the neighboring medicinal slice in the sub-model. If so, it determines that there is an abnormality in the capture, indicating that the pharmacist has captured the medicinal slice adjacent to the target medicinal slice.

[0047] Further, see Figure 3 The real-time image includes images of the target medicinal slice and its neighboring slices in the distillation chart. The identification and verification of the real-time image also includes the following steps: Perform multi-region search on real-time images to obtain window images corresponding to different medicinal slices; Identify the types of medicinal slices in the window image and their distribution location in the real-time image, and compare and analyze them with the sub-model; If the types of medicinal slices in the window image and their distribution positions in the real-time image do not correspond to those in the sub-model, it is determined that there is an anomaly in the current capture operation.

[0048] The scope of the real-time image is not limited to the target medicinal slice itself, but also includes a larger field of view of the target medicinal slice and several neighboring medicinal slices recorded in the doupu (a type of medicinal container). For example, the image should include the target medicinal container (slice) and the adjacent medicinal containers (slices) on its left, right, top, and bottom. Therefore, the sub-model recognizes and expects to see the target medicinal slice in a background environment composed of several specific neighboring medicinal slices in the verified real-time image.

[0049] In other words, when the system calls the sub-model corresponding to the current target medicinal slice to verify the acquired real-time image, it performs a multi-region search and contextual comparison analysis process.

[0050] The specific process is as follows: First, the system performs a multi-region search on the real-time image, which is usually achieved through a detection window of a preset size. The window slides on the image with a certain horizontal and vertical step size (i.e., offset), traversing different regions of the image. Each time it slides to a new position, the image area covered by the window is captured to form an independent window image. In this way, the system decomposes a complete real-time image into a series of window image sequences that cover local details, aiming to capture various independent objects that may exist in the image, such as different slices of medicine, the operator's hand, tools, etc.

[0051] Subsequently, for each acquired window image, the system identifies the main types of objects contained within the window, such as the target medicinal slice Astragalus membranaceus, the adjacent medicinal slice Angelica sinensis, the background medicine container, or foreign objects such as human hands or scissors. At the same time, the system records the precise distribution position of the window image in the original real-time image, such as the coordinates of its center point.

[0052] Next, the system treats the recognition results of the aforementioned window images (the types of medicinal slices / objects and their location distribution) as a whole and compares them with the expected criteria embedded in the currently invoked sub-model. This expected criterion, derived from the training data of the sub-model, essentially defines what objects should appear in the real-time image and their spatial layout relationships when the image is correctly captured. For example, for the target medicinal slice Astragalus membranaceus, the sub-model expects: Astragalus membranaceus should be primarily identified in the central region of the image; its fixed neighboring medicinal slices, such as Angelica sinensis and Codonopsis pilosula, should be stably identified in specific edge regions such as the upper left and upper right; and no human hands or metal instruments should appear in the medicine drawer area.

[0053] Based on this comparison, the system makes a comprehensive judgment: if the identified types of medicinal slices and their spatial distribution in the image highly match the standard layout expected by the sub-model, it is judged as normal; conversely, if the recognition result of the window image does not match the expectation, for example: licorice is identified in the position where the expected adjacent medicinal slice angelica should be; or a human hand is identified instead of the target medicinal slice astragalus in the central area of ​​the image; or an unknown object not labeled in the training set is identified in the medicine cabinet area, as long as at least one of the types of medicinal slices or their distribution positions in the window image does not correspond to the expected standard embedded in the sub-model, the system can comprehensively judge that there is a foreign object or the grasping background is wrong, thereby determining that the current grasping operation is abnormal.

[0054] This invention decomposes and analyzes images through multi-region search, which can capture subtle local anomalies that may be overlooked in whole-image recognition. Combined with dual comparison standards of type and location, the system can not only detect obvious errors such as picking the wrong medicine, but also keenly identify a variety of potential problems such as background medicine errors caused by the picking position shift, a small amount of other medicine spilled and mixed in, and the operating tool accidentally entering the frame. This greatly reduces the possibility of missed detection and significantly improves the recognition accuracy and reliability.

[0055] In some implementations, see Figure 3 The spatial coordinates of the beam spectrum include horizontal and vertical coordinates, and the real-time images include horizontal and vertical images; When performing recognition and verification on real-time images, both horizontal and vertical images are compared and analyzed simultaneously.

[0056] In this embodiment, in order to capture information from the scene more comprehensively and in a more three-dimensional way, the data collection and verification logic has been enhanced in multiple dimensions.

[0057] First, when constructing the medicine slice catalog, the recorded spatial coordinates are three-dimensional, at least explicitly including horizontal coordinates, such as the X and Y axis positions on the medicine cabinet plane, which can correspond to the row and column numbers; and vertical coordinates, such as the height or layer number in the Z axis direction. This three-dimensional coordinate system precisely defines the unique position of each medicine slice in the three-dimensional space of the medicine cabinet.

[0058] Secondly, when grasping the target medicinal slices, the image acquisition device is configured, or operated in a guided manner, to acquire real-time images from at least two different angles, specifically including: Horizontal image: The acquisition device is aimed at the front of the medicine cabinet in a roughly horizontal direction to take a picture. The image can clearly show the appearance of the medicine slices in the target medicine cabinet and its left and right adjacent medicine cabinets, focusing on the side shape, color and horizontal proximity of the medicine slices.

[0059] Vertical image (or top-down image): The acquisition device takes a picture of the medicine slices being grabbed and the medicine container they are in from above or diagonally above. The image can clearly show the texture of the surface of the medicine slices, the approximate area of ​​the grabbed dose stack, and other medicine slices that may exist vertically below or inside.

[0060] During the identification and verification process, the control unit executes a multi-view analysis process. The system calls the corresponding sub-model based on the three-dimensional coordinates of the current target medicinal slice. Subsequently, the sub-model is applied to the analysis of both the horizontal and vertical images. The system extracts features from these two images and performs identification. For example, through the above-mentioned multi-region search method, two sets of verification results are obtained regarding the types, quantities, and spatial distribution of medicinal slices.

[0061] Then, the control unit compares the recognition result of the horizontal image with the expected scene trained based on the horizontal viewpoint in the sub-model, and at the same time compares the recognition result of the vertical image with the expected scene trained based on the top-down viewpoint.

[0062] The final judgment is based on the intersection and logical combination of these two comparisons. For example, for the capture operation to be judged as normal, it must simultaneously meet the following conditions: the horizontal image verification is passed and the vertical image verification is passed, that is, the target medicinal slice is correct, the horizontal neighbors are correct, the surface features of the target medicinal slice are correct, and no unexpected medicinal slices are mixed in from above. If the image verification of either view fails, or there is an inexplicable contradiction in the verification results of the two views, such as the horizontal image being identified as medicinal slice A and the top view being identified as medicinal slice B, the system judges that there is an anomaly.

[0063] By combining image information from two orthogonal dimensions, horizontal and vertical, this invention enables the system to construct a more three-dimensional scene understanding. The horizontal view verifies who the neighbors are, preventing the wrong medicine container from being grabbed; the top view verifies what is in the hand or in the container, effectively detecting non-target medicinal slices brought out from the lower layer due to grabbing too deep, or identifying impurities mixed in on the surface. This multi-view cross-verification greatly reduces the visual blind spots or deception that may exist in single-angle shooting, making anomaly detection more comprehensive and reliable, and further improving recognition accuracy and reliability.

[0064] Furthermore, when a medicine cabinet is designed to accommodate multiple medicinal herbs horizontally side-by-side within a single drawer, the spatial coordinates recorded in the drawer not only need to distinguish between different drawers but also need to be precise down to the different sections within the same drawer. For example, a long, narrow drawer is divided into three compartments—left, middle, and right—by a partition, storing medicinal herbs A, B, and C respectively. In this case, the adjacent medicinal herbs for herb B include herb A to its left and herb C to its right within the same drawer.

[0065] The image acquisition and verification process therefore has the following characteristics: When capturing the target medicinal slice (e.g., slice B in the medicine cabinet), the horizontal image (front view) can clearly present the entire front of the medicine cabinet, thereby simultaneously capturing the appearance and relative position of the target medicinal slice B and its vertically adjacent slices D and E. The vertical image (top view or oblique top view) can present a top view of the inside of the medicine cabinet, showing the appearance and relative position of its horizontally adjacent slices A and C.

[0066] During the identification and verification process, the sub-model invoked by the system includes the features of the target medicinal slice B and its horizontally adjacent medicinal slices A and C fixed in the same medicine container, as well as the features of the target medicinal slice B and its vertically adjacent medicinal slices D and E. The system analyzes both the horizontal and vertical images simultaneously.

[0067] Horizontal image analysis: Verify whether the shape of the sliced ​​medicine at the captured position matches the target sliced ​​medicine B from a frontal view, and whether the sliced ​​medicines appearing on its left and right sides are indeed the expected D and E.

[0068] Vertical image analysis: Verify whether the surface feature of the medicinal slice in the target cell is B when viewed from above, and whether the medicinal slices appearing on its left and right sides are indeed the expected A and C.

[0069] Comprehensive judgment: The system determines that the capture is normal only if the recognition results of both the horizontal and vertical images are consistent with the spatial visual relationship expected by the sub-model, i.e., the horizontal sequence is correct and the vertical sequence is correct. If any discrepancy is detected in any image, such as recognizing a similar object G at position B in the horizontal image, or finding a medicinal slice F in the top view, an anomaly warning will be triggered.

[0070] In some implementations, see Figure 2 and Figure 3 The process involves retrieving target medicinal herbs based on the prescription, including: planning the dispensing path and order of herbs based on the prescription and prescription chart, and retrieving herbs in that order.

[0071] Specifically, when the system receives a prescription, the control unit simultaneously retrieves the details of all the medicinal herbs listed in the prescription and the stored medicinal herb database. The system first parses the prescription to obtain a list of all target medicinal herbs to be picked up. Then, based on the database, the system queries the precise spatial coordinates of each medicinal herb in the list, such as the zone, row, and column. Then, based on the distribution of these coordinates in the three-dimensional space of the pharmacy, the system runs a path planning algorithm to generate one or more optimal or suboptimal physical movement paths. The optimization objectives include: minimizing the total movement distance, minimizing the total time, or avoiding repeated round trips. At the same time, the algorithm plans the matching order of picking up the herbs based on this optimal path.

[0072] For example, a prescription requires picking up herbal pieces A, B, C, and D. Calculations show that if the dispensing cart moves in the order of A, C, B, and D, it needs to make a long backtracking trip. However, if it moves in the order of A, B, C, and D, it forms a smooth circular route, significantly shortening the total distance traveled. The system will then determine A, B, C, and D as the recommended or mandatory order for this dispensing.

[0073] Once the planning is complete, the human-computer interaction terminal will clearly guide the operation according to this optimized sequence. For example, the screen will first highlight the first step: Please go to [01 area - 03 row - 05 column] to grab slice A (target dose: 10g), with arrows indicating the direction; after slice A is verified by image and / or weighed and confirmed to be correct, the system will automatically update the interface to the second step: Please go to [01 area - 03 row - 06 column] to grab slice B (target dose: 5g); the operator can complete the entire dispensing process by following the step-by-step guidance of the system.

[0074] In some implementations, see Figure 2 and Figure 3 The adjustment methods also include: if the warning is confirmed to be a false alarm, adjusting the adjacent layout of the relevant medicinal slices in the formulary and / or optimizing the corresponding sub-model.

[0075] In this embodiment, the system not only has an early warning function, but also includes a recording and learning module. When an early warning is triggered, the operator performs on-site verification, and the verification result is fed back to the system through the human-computer interaction terminal. It mainly includes two types: confirmed as a real error, or confirmed as a false alarm. The core of this embodiment lies in handling false alarm situations.

[0076] If a pharmacist confirms that the warning is a false alarm, meaning that the image recognition system mistakenly identified a correct capture as abnormal, the system will record this event and associate it with the current task context, storing the target medicinal slice, the called sub-model, real-time image, etc., to form a false alarm case.

[0077] Based on the analysis of false positive cases, the system will implement one or two of the following optimization measures: Adjusting the adjacent layout of related medicinal slices in the prescription: Through analysis of multiple false alarm records, the system may find that the warning for a certain target medicinal slice is often because a certain fixed adjacent medicinal slice is too visually similar to it, making it difficult for the sub-model to distinguish. At this time, the system prompts the pharmacy administrator to adjust the physical placement of the medicine cabinet, that is, to modify the prescription.

[0078] For example, visually easily confused medicinal slices A and B are removed from their adjacent positions so that they are no longer neighbors defined in the Doupu (a traditional Chinese medicine classification system). After adjustment, the Doupu database is updated. Subsequently, sub-models for medicinal slices A and B will be trained or invoked based on the new neighbor relationships. Since the interference items in the background environment are removed, the false alarm rate is significantly reduced.

[0079] Alternatively, optimize the corresponding sub-model: This is a method of improvement at the algorithm model level. False positives indicate that the current sub-model's generalization ability is insufficient for certain specific scenarios (such as specific lighting, specific shapes of medicinal slices, and new mixed backgrounds). The system initiates the model optimization process, collecting real-time images from false positive cases related to the target medicinal slice in this case and in the past. After the pharmacist confirms their correctness, these images are added to the training set of the sub-model as new positive samples.

[0080] Alternatively, more diverse images of the target herbal slices in the vicinity can be acquired to expand the original training set. The updated training set can then be used to retrain or fine-tune the original sub-model to obtain a new sub-model, which can then replace the original model.

[0081] Furthermore, these two optimization methods can be implemented independently or in combination. For example, for systemic confusion caused by unreasonable layout, the bucket spectrum should be adjusted first; for model defects caused by insufficient samples, supplementary data should be used to optimize the model.

[0082] Through this closed loop of early warning, feedback, and optimization, the system can continuously learn from actual use, dynamically adapt to changes in the pharmacy environment and operating habits, and ultimately achieve a continuous reduction in false alarm rate and a steady improvement in the overall reliability of the system.

[0083] A second aspect of the invention is described herein. Figure 4 This invention provides a traditional Chinese medicine dispensing vehicle for implementing the above-mentioned method of dispensing traditional Chinese medicine decoction pieces, which is a hardware entity that integrates intelligent verification function into a mobile platform.

[0084] The dispensing cart includes a movable body, an image acquisition device, and a control unit. The body is a wheeled cart, which allows pharmacists to move flexibly between medicine cabinets. The body is equipped with a load-bearing platform or container placement area to carry medicine trays, packaging bags, or dispensing containers used in the dispensing process.

[0085] The image acquisition device is fixedly installed on the vehicle body. Its installation position and angle are specially designed to ensure that when the operator stands next to the vehicle body to grasp the medicine, the target medicinal slices and their image in the target medicine compartment area can be clearly and stably captured.

[0086] The image acquisition device includes a high-definition camera, comprising a first camera for acquiring target medicinal slices, a second camera for acquiring horizontal images of the target medicinal slices and their location in the target medicine drawer area, and a third camera for acquiring vertical images of the target medicinal slices and their location in the target medicine drawer area.

[0087] The control unit includes an embedded computer or industrial controller, which is located inside or outside the vehicle body. The control unit can be directly integrated with the vehicle body, or it can maintain a communication connection with the vehicle body via a wireless network and be deployed on a remote server.

[0088] The control unit is configured to execute the following program logic: Storage or access to medicinal slice catalogs: It internally stores or can access the pharmacy's medicinal slice catalog database in real time via the network. This database contains the spatial coordinate information of all medicinal slices.

[0089] Based on the prescription and the prescription data, the current target herbal medicine is determined: After receiving or loading an electronic prescription, the control unit parses the prescription content and, in conjunction with the prescription data, determines the specific target herbal medicine to be picked up in the current dispensing step and its precise location in the medicine cabinet.

[0090] Sub-model is invoked for recognition and verification: After the operator completes a grabbing action and triggers image acquisition, the control unit calls a pre-trained image recognition sub-model corresponding to the target herbal slice from the local or cloud model library according to the target herbal slice information. Then, the sub-model is used to analyze and compare the real-time image transmitted by the image acquisition device and execute the verification process as described in the aforementioned method implementation.

[0091] The control unit makes a judgment based on the verification results of image recognition. Once an anomaly is determined, a control command is generated to drive the warning device (such as a buzzer or warning light) installed on the vehicle body to issue an audible and visual alarm. At the same time, the warning information is also sent to the human-machine interaction terminal for display.

[0092] The Chinese herbal medicine dispensing cart of this embodiment highly integrates image acquisition, intelligent computing and early warning functions into a mobile cart, enabling it to follow the pharmacist and provide on-the-go verification services in front of any medicine cabinet. In actual complex dispensing scenarios, it achieves accuracy and reliability far exceeding that of general recognition models, effectively solving the technical problem of easy misidentification of similar-looking herbal medicine pieces, and significantly improving recognition accuracy and reliability.

[0093] In some implementations, see Figure 4 In addition to image recognition verification, the dispensing vehicle also integrates a weighing module. The weighing module (such as an electronic scale) is fixedly installed on the vehicle body, and its weighing plate or tray is used to temporarily place the grabbed beverage pieces for accurate weighing.

[0094] The control unit is also configured to: receive the weighing result signal from the weighing module in real time, obtain the theoretical dose value corresponding to the current target herbal piece from the currently processed electronic prescription, compare the actual weighing result with the theoretical dose of the prescription, and calculate the absolute deviation or relative percentage deviation between the two; preset a reasonable dose deviation threshold (e.g., ±5%), and when the calculated deviation exceeds this threshold, it is determined that the dose deviation exceeds the limit. At this time, the control unit records the abnormality and also controls the early warning device to issue an early warning. This early warning can be independent of the image verification early warning or it can be combined with the image verification result for a comprehensive early warning.

[0095] Specifically, after the operator picks up the medicinal slices, they first place them on the weighing module for weighing, and the control unit completes the dosage comparison. If the dosage is qualified, the medicinal slices are then moved to the image acquisition area to acquire images for variety verification. The control unit combines the two verification results to make a final judgment.

[0096] In some implementations, see Figure 4 The dispensing vehicle also includes a human-computer interaction terminal, which is installed on the vehicle body to display prescription information, dispensing guidance information, and early warning information.

[0097] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for dispensing traditional Chinese medicine decoction pieces, characterized in that, comprising the following steps: compiling a drawer spectrum for prepared herbal pieces, wherein the drawer spectrum records the spatial coordinates of each prepared herbal piece stored in the herbal cabinet; grabbing a target prepared herbal piece according to a prescription, and collecting a real-time image of the target prepared herbal piece; calling a sub-model corresponding to the target prepared herbal piece in an image recognition model according to the drawer spectrum coordinates of the current target prepared herbal piece, to perform recognition and verification on the real-time image; wherein the sub-model comprises images of the target prepared herbal piece and the prepared herbal pieces adjacent to the target prepared herbal piece in the drawer spectrum; judging whether the current grabbing has an abnormality, and sending out early warning information when an abnormality is determined.

2. The method for dispensing traditional Chinese medicine decoction pieces according to claim 1, characterized in that, said image recognition model comprises a plurality of sub-modules for different target prepared herbal pieces, and the steps for training the sub-models comprise: collecting sample images of the target prepared herbal piece and the prepared herbal pieces adjacent to the target prepared herbal piece in the drawer spectrum; performing labeling processing on the sample images to form a training set; associating the prepared herbal pieces in the training set with the spatial coordinates thereof in the drawer spectrum, and training to obtain the sub-model corresponding to the target prepared herbal piece.

3. The method for dispensing traditional Chinese medicine decoction pieces according to claim 2, characterized in that, said labeling processing on the sample images comprises: cropping the sample images according to preset grid regions to form a plurality of regional images, wherein different regional images respectively correspond to different prepared herbal pieces; selecting a plurality of regional images comprising the target prepared herbal piece, and labeling the types of prepared herbal pieces contained in each regional image.

4. The method for dispensing traditional Chinese medicine decoction pieces according to any one of claims 1-3, characterized in that, performing recognition and verification on the real-time image comprises: performing comparative analysis on the real-time image and the sub-model corresponding to the target prepared herbal piece; if the prepared herbal piece in the real-time image does not correspond to the sub-model, judging that the current grabbing operation has an abnormality; preferably, the real-time image comprises images of the target prepared herbal piece and the prepared herbal pieces adjacent to the target prepared herbal piece in the drawer spectrum, and performing recognition and verification on the real-time image further comprises: performing multi-region search on the real-time image to obtain window images corresponding to different prepared herbal pieces; recognizing the types of prepared herbal pieces in the window images and the distribution positions thereof in the real-time image, and performing comparative analysis with the sub-model; if the types of prepared herbal pieces in the window images and the distribution positions thereof in the real-time image do not correspond to those of the sub-model, judging that the current grabbing operation has an abnormality.

5. The method for dispensing traditional Chinese medicine decoction pieces according to any one of claims 1-4, characterized in that, the spatial coordinates of the drawer spectrum comprise horizontal coordinates and vertical coordinates, and the real-time image comprises a horizontal image and a vertical image; when performing recognition and verification on the real-time image, the horizontal image and the vertical image are comparatively analyzed at the same time.

6. The method for dispensing traditional Chinese medicine decoction pieces according to any one of claims 1-5, characterized in that, said grabbing the target prepared herbal piece according to the prescription comprises: planning a dispensing path and a grabbing sequence according to the prescription and the drawer spectrum, and grabbing according to the sequence.

7. The method for dispensing traditional Chinese medicine decoction pieces according to any one of claims 1-6, characterized in that, further comprising: if the early warning is confirmed to be a false alarm, adjusting the adjacent layout of related prepared herbal pieces in the drawer spectrum, and / or optimizing the corresponding sub-model.

8. A traditional Chinese medicine dispensing cart for implementing the dispensing method of any one of claims 1-7, characterized in that, comprising: a vehicle body; an image acquisition device, arranged on the vehicle body and configured to collect a real-time image of a target prepared herbal piece; a control unit, arranged on the vehicle body or communicatively connected with the vehicle body, wherein the control unit is configured to: store or access a prepared herbal piece drawer spectrum; determine the current target prepared herbal piece according to the prescription and the drawer spectrum; call the sub-model corresponding to the current target prepared herbal piece to perform recognition and verification on the real-time image; control an early warning device to perform abnormality early warning according to a verification result.

9. The dispensing cart for traditional Chinese medicine decoction pieces according to claim 8, characterized in that, further comprising: a weighing module, arranged on the vehicle body and configured to weigh the grabbed prepared herbal piece; said control unit is further configured to: compare a weighing result with a prescription dose, and control to give an early warning when the dose deviation exceeds a limit.

10. The dispensing cart for traditional Chinese medicine decoction pieces according to claim 8 or 9, characterized in that, It also includes a human-computer interaction terminal, which is installed on the vehicle body and is used to display prescription information, dispensing guidance information and warning information.

Images