Image recognition system, image recognition method and image capturing subsystem

Abstract

An image identification system is suitable for identifying a plurality of containers. The system comprises an image capturing element, a bar code identification module, a container feature identification module and a comparison module. The image capturing element is used for capturing a plurality of images, wherein each image comprises images of a plurality of containers at different angles. The bar code identification module is used for obtaining a first comparison result by identifying bar codes on a plurality of containers in a plurality of images. The container feature identification module is used for obtaining a second comparison result by identifying appearance features of a plurality of containers in the plurality of images. The comparison module is used for comparing whether the first comparison result is consistent with the second comparison result to generate a final result.

Description

Technical Field

[0001] This invention relates to an image recognition system, an image recognition method, and an image acquisition subsystem, particularly an image recognition system, an image recognition method, and an image acquisition subsystem suitable for recognizing containers. Background Technology

[0002] Current container placement devices (such as, but not limited to, medicine bottle placement devices) typically utilize two internal slides to hold multiple containers. The relative movement of these slides in the same direction allows workers to replenish containers from one location and perform tasks such as container image recognition or dispensing at another. However, when using a camera to image the containers, the varying distances between the camera and different slides necessitate frequent adjustments to the image position or camera focus to maintain optimal image quality. Therefore, any change in container placement necessitates camera repositioning, resulting in significant time costs. Furthermore, the containers may contain toxic substances, posing a safety risk due to the potential for toxic gas leakage during replenishment.

[0003] Therefore, a novel image recognition system, image recognition method, and image acquisition subsystem are needed to improve the above problems. Summary of the Invention

[0004] This invention provides an image recognition system suitable for identifying multiple containers. The system includes an image capturing element, a barcode recognition module, a container feature recognition module, and a comparison module. The image capturing element acquires multiple images, each containing images of multiple containers at different angles. The barcode recognition module obtains a first comparison result by recognizing barcodes on multiple containers in the multiple images. The container feature recognition module obtains a second comparison result by recognizing appearance features of multiple containers in the multiple images. The comparison module compares the first and second comparison results to generate a final result.

[0005] This invention also provides an image recognition method applicable to recognizing multiple containers. The method is executed by an image recognition system, which includes an image capturing element, a barcode recognition module, a container feature recognition module, and a comparison module. The method includes the steps of: acquiring multiple images through the image capturing element, wherein each image contains images of multiple containers at different angles; using the barcode recognition module to identify barcodes on multiple containers in the multiple images to obtain a first comparison result; using the container feature recognition module to identify appearance features of multiple containers in the multiple images to obtain a second comparison result; and using the comparison module to compare whether the first comparison result and the second comparison result match to generate a final result.

[0006] The present invention also provides an image acquisition subsystem, comprising a frame assembly, a driving element, and an image acquisition element. The frame assembly comprises multiple support frames arranged in an N-plane, where N is a positive integer greater than or equal to 4, and each of the multiple support frames has multiple platforms, each of the multiple platforms being used to hold a container. The driving element is used to rotate the multiple platforms to rotate the container. Attached Figure Description

[0007] Figure 1A A system architecture diagram of an image recognition system according to an embodiment of the present invention is shown;

[0008] Figure 1B A schematic diagram of an image-capturing subsystem according to an embodiment of the present invention is shown;

[0009] Figure 1C A partially enlarged view of an image-capturing subsystem according to an embodiment of the present invention is shown;

[0010] Figure 2A A flowchart illustrating the steps of a drug preparation process according to an embodiment of the present invention is shown.

[0011] Figure 2B A flowchart illustrating the steps of a drug preparation process according to another embodiment of the present invention is shown.

[0012] Figure 3A A schematic diagram of a clamping device and a container according to an embodiment of the present invention is shown;

[0013] Figure 3B A schematic diagram of a clamping device according to an embodiment of the present invention is shown, which is disposed on a platform.

[0014] Figure 4 A partially exploded view of a maintenance door according to an embodiment of the present invention is shown;

[0015] Figure 5 A schematic diagram illustrating the application of an image-capturing subsystem according to an embodiment of the present invention is shown;

[0016] Figure 6 A main flowchart of an image recognition method according to an embodiment of the present invention is shown;

[0017] Figure 7 A detailed flowchart of an image recognition method according to an embodiment of the present invention is shown;

[0018] Figure 8 A detailed flowchart of an image recognition method according to an embodiment of the present invention is shown;

[0019] Figure 9 A detailed flowchart of an image recognition method according to an embodiment of the present invention is shown.

[0020] Figure label:

[0021] Image recognition system 1;

[0022] Container 2;

[0023] Image acquisition subsystem 3;

[0024] Image correction module 4;

[0025] Barcode recognition module 5;

[0026] Container feature identification module 6;

[0027] Comparison Module 7;

[0028] Database 8;

[0029] Clamping device 9;

[0030] Frame group 10;

[0031] Drive element 20;

[0032] Image capturing element 30;

[0033] Barcode database 81;

[0034] Frontal image database 82;

[0035] Color database 83;

[0036] Size database 84;

[0037] Detailed feature database 85;

[0038] Artificial intelligence model database 86;

[0039] Image segmentation module 70;

[0040] Support frame 11;

[0041] Platform 12;

[0042] Shelf 13;

[0043] Base 21;

[0044] Sub-driving element 22;

[0045] Platform plane 121;

[0046] Stage rotation axis 122;

[0047] Hole 131;

[0048] Bearings 123 and 222;

[0049] Time-return belt 223;

[0050] Positioning component 124;

[0051] Positioning hole 91;

[0052] Image capture range Ri;

[0053] Isolation door 40;

[0054] Ontology 41;

[0055] Window 42;

[0056] First window plate 43A;

[0057] Second window plate 43B;

[0058] Handle 44;

[0059] Door lock 45;

[0060] Robotic arm 60;

[0061] Groove 92;

[0062] Steps A1~A5, B1~B6, S1~S6, S100~S160, S200~S280, S300~S360;

[0063] Cavity 50. Detailed Implementation

[0064] The present invention will now be described in detail with reference to exemplary embodiments thereof, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element references are used in the drawings and description to denote the same or similar parts.

[0065] Throughout this specification and claims, certain terms are used to refer to specific components. Those skilled in the art will understand that sensing device manufacturers may use different names to refer to the same components. This document is not intended to distinguish between components that have the same function but different names. In the following specification and claims, words such as “containing,” “comprising,” and “including” are open-ended terms and should therefore be interpreted as “containing but not limited to…”.

[0066] The terms “approximately,” “substantially,” or “roughly” are generally interpreted as being within 10% of a given value or range, or as being within 5%, 3%, 2%, 1%, or 0.5% of a given value or range.

[0067] The ordinal numbers used in the specification and claims, such as "first," "second," etc., to modify elements, do not in themselves imply or represent any prior ordinal number of that element (or those elements), nor do they represent the order of one element with another, or the order of manufacturing processes. The use of these ordinal numbers is solely to clearly distinguish one named element from another element with the same name. The claims and specification may not use the same terminology; therefore, a first element in the specification may be a second element in the claims.

[0068] In this invention, the terms "given range is from the first value to the second value" and "given range falls within the range of the first value to the second value" indicate that the given range includes the first value, the second value, and other values ​​in between.

[0069] Furthermore, the image recognition system disclosed in this invention can be applied to electronic devices themselves, applications of electronic devices, or manufacturing processes of electronic devices. Electronic devices may include automated equipment, clamping devices, object-grabbing devices on mobile platforms, computing devices, mechanical equipment, drug dispensing equipment, exposure devices, printing devices, three-dimensional printing devices, automotive devices, image-capturing devices, assembly devices, backlight devices, antenna devices, splicing devices, touch electronic devices, curved electronic devices, or free-shape electronic devices, but are not limited thereto. Display devices may include, for example, liquid crystal, light-emitting diode, fluorescence, phosphorescence, other suitable display media, or combinations thereof, but are not limited thereto. Display devices may be non-self-emissive or self-emissive. Antenna devices may be liquid crystal type antenna devices or non-liquid crystal type antenna devices. Sensing devices may be sensing capacitance, light, heat, or ultrasound, but are not limited thereto. The splicing device may include, for example, a display splicing device or an antenna splicing device, but is not limited thereto. It should be noted that the electronic device may be any of the aforementioned arrangements and combinations, but is not limited thereto. Furthermore, the electronic device may be a bendable or flexible electronic device. It should be noted that the electronic device may be any of the aforementioned arrangements and combinations, but is not limited thereto. Furthermore, the shape of the electronic device may be rectangular, circular, polygonal, with curved edges, or other suitable shapes. The electronic device may have peripheral systems such as a drive system, control system, light source system, shelf system, etc., to support the display device, antenna device, or splicing device. The electronic device may include, for example, electronic components, liquid crystal, light-emitting diodes, quantum dots (QD), fluorescence, phosphorescence, other suitable display media, or combinations of the above materials, but is not limited thereto. Electronic components may include passive and active components, such as capacitors, resistors, inductors, diodes, transistors, etc. Diodes may include light-emitting diodes or photodiodes. Light-emitting diodes may include, for example, organic light-emitting diodes (OLEDs), mini LEDs, micro LEDs, or quantum dot LEDs (including QLEDs and QDLEDs), light-emitting diodes for flexible displays, or other suitable materials, or combinations thereof, but are not limited thereto.

[0070] It should be understood that the following embodiments can be modified by replacing, recombining, or mixing features from multiple different embodiments to complete other embodiments without departing from the spirit of the invention. Features between embodiments can be arbitrarily mixed and combined as long as they do not violate the spirit of the invention or conflict with it.

[0071] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It is understood that these terms, for example, as defined in commonly used dictionaries, should be interpreted as having a meaning consistent with the relevant art and the background or context of this invention, and should not be interpreted in an idealized or overly formal manner, unless specifically defined in the embodiments of this invention.

[0072] Furthermore, the term "adjacent" in the specification and claims is used to describe objects that are close to each other, and there may be contact or no contact between adjacent objects.

[0073] Furthermore, descriptions such as "when..." or "...when" in this invention refer to "at present, before, or after," and are not limited to simultaneous occurrences; this is stated in advance. Descriptions such as "set on..." in this invention indicate the corresponding positional relationship between two elements, and do not limit whether the two elements are in contact, unless specifically limited; this is stated in advance. Moreover, when this invention describes multiple functions, the use of the word "or" between functions indicates that the functions can exist independently, but does not preclude the simultaneous existence of multiple functions.

[0074] Figure 1A A schematic diagram of an image recognition system 1 according to an embodiment of the present invention is shown, wherein the image recognition system 1 is suitable for recognizing multiple containers 2 and can be incorporated into an electronic device. Figure 1A As shown, the image recognition system 1 includes at least an image acquisition subsystem 3, a barcode recognition module 5, a container feature recognition module 6, and a comparison module 7. Furthermore, the image recognition system 1 may also include an image correction module 4, and may further include a database 8. In one embodiment, the image acquisition subsystem 3 may include a frame assembly 10, a driving element 20, and an image acquisition element 30. In one embodiment, the database 8 may include at least one of a barcode database 81, a frontal image database 82, a color database 83, a size database 84, and a detail feature database 85, and is not limited thereto. Furthermore, the image recognition system 1 may be used in conjunction with a clamping device 9, wherein the clamping device 9 can be used to clamp the container 2, and the clamping device 9 can be mounted on the frame assembly 10, thereby placing the container 2 on the frame assembly 10, and is not limited thereto. Furthermore, in one embodiment, the image recognition system 1 may also include an image segmentation module 70.

[0075] The image-capturing subsystem 3's image-capturing element 30 can acquire multiple images, and each image contains at least one image of the container 2 at different angles. Here, "different angles" refers to the rotation angle of the container 2 itself in the horizontal direction, for example, simultaneously referencing... Figure 1B Each container 2 can rotate relative to the frame assembly 10 around the Z-direction, and the imaging element 30 can capture images of the container 2 at different rotation angles, but is not limited to this. In one embodiment, the imaging element 30 can capture an image every 5 degrees of rotation of the container 2, thus the imaging element 30 will acquire 72 images for one container 2, but is not limited to this. The barcode recognition module 5 can be used to recognize the barcode on the container 2 in the image, thereby obtaining a first comparison result. The container feature recognition module 6 can be used to recognize the appearance features of the container 2 in the image, thereby obtaining a second comparison result. The comparison module 7 can be used to compare whether the first comparison result and the second comparison result match, thereby generating a final result. In addition, in one embodiment, after the imaging element 30 acquires the image, the image segmentation module 70 can be used to segment each container 2 in the image to form a container image, and the barcode recognition module 5 and the container feature recognition module 6 can individually recognize the container image of each container 2, but are not limited to this. Therefore, the image recognition system 1 can be used to identify the type of container 2 or to determine the type of contents contained in container 2, and is not limited thereto.

[0076] Next, the details of each component will be explained.

[0077] First, let me explain the details of the image acquisition subsystem 3. Figure 1B A schematic diagram of an image-capturing subsystem 3 according to an embodiment of the present invention is shown. Figure 1C A partially enlarged view of the image-capturing subsystem 3 according to an embodiment of the present invention is shown, and please also refer to... Figure 1A .in Figure 1C It corresponds Figure 1B A local region A in the middle.

[0078] Regarding frame 10, as follows Figure 1B and 1CAs shown, the frame assembly 10 of the image acquisition subsystem 3 may include a plurality of support frames 11, which may be arranged in an N-hedron, where N is a positive integer greater than or equal to 4 (4 ≤ N). In one embodiment, N may be, for example, an even number greater than or equal to 4, and the plurality of support frames 11 may be arranged in a tetrahedron, hexahedron, or octahedron, etc., and is not limited thereto. Each support frame 11 may have a plurality of platforms 12, and more specifically, each support frame 11 may have a plurality of shelves 13, and each shelf 13 is provided with a plurality of platforms 12, wherein the shelves 13 of each support frame 11 may be arranged along, for example, the Z direction, wherein adjacent shelves 13 are separated from each other along the Z direction, and the platforms 12 on each shelf 13 are arranged, for example, along a direction substantially perpendicular to the Z direction (e.g., the X direction or the Y direction, and is not limited thereto). Each platform 12 may be used to accommodate a container 2, wherein the container 2 may be accommodated on the platform 12 by means of a clamping device 9.

[0079] Regarding the drive element 20, in one embodiment, the drive element 20 may include a main shaft (not shown). The main shaft of the drive element 20 may be pivotally connected to a base 21 and fixed to the frame assembly 10. Therefore, when the main shaft of the drive element 20 rotates relative to the base 21, the drive element 20 may cause the frame assembly 10 to rotate relative to the base 21, for example, causing one of the support frames 11 of the frame assembly 10 to face the image-capturing element 30, but not limited thereto. Thus, the details of the frame assembly 10 can be understood.

[0080] In addition, such as Figure 1C As shown, the drive element 20 may include multiple sub-drive elements 22, each of which can be used in conjunction with a platform 12 on a shelf 13. In one embodiment, the platform 12 may have a platform plane 121 and a platform pivot 122, wherein the platform pivot 122 may be located directly below the platform plane 121, but is not limited thereto. The shelf 13 may include multiple holes 131, the size of which may be larger than the platform pivot 122 and smaller than the platform plane 121, so that the platform pivot 122 can pass through the holes 131 and cause the platform plane 121 to rest against the shelf 13, but is not limited thereto. The sub-drive element 22 may be, for example, a timing wheel, and may be used in conjunction with a bearing 222, wherein, in, for example, in the Z direction, the platform pivot 122 may pass through the hole 131 of the shelf 13 via a bearing 123, and be combined with the sub-drive element 22 below the shelf 13 via another bearing 222. In addition, a timing belt 223 can be fitted onto multiple sub-drive elements 22 under the same shelf 13.

[0081] Please refer to the following at the same time Figure 1C and 2A ,in Figure 2A A schematic diagram of a clamping device 9 and a container 2 according to an embodiment of the present invention is shown. Figure 1C and 2A As shown, each platform 12 may have at least one positioning element 124 on its platform plane 121, and the clamping device 9 may have at least one positioning hole 91 corresponding to the at least one positioning element 124. Therefore, the clamping device 9 can be disposed on the platform 12 and positioned. In one embodiment, when the orientation of the positioning element 124 on the platform plane 121 of each platform 12 is set to be consistent, each clamping device 9 can be disposed on the platform 12 in a consistent orientation.

[0082] Please refer to the following at the same time Figures 1B to 2B ,in Figure 2B A schematic diagram of a clamping device 9 disposed on a platform 12 according to an embodiment of the present invention is shown. Figure 2B As shown, in one embodiment, when the clamping device 9 clamps the container 2 and is placed on the platform 12, at least one platform shaft 122 of the platform 12 protrudes from the sub-drive element 22 and can be connected to an external power source (not shown). Therefore, when the external power source provides power to rotate the platform shaft 122, the sub-drive element 22 fixed to the platform shaft 122 can rotate relative to the hole 131 of the shelf 13 and drive other sub-drive elements 22 to rotate together through the timing belt 223. The rotation of the platform shaft 122 and the sub-drive elements 22 can also drive the platform 12 to rotate together, so that the clamping device 9 and the container 2 placed on the platform 12 rotate, and this is not limited to this. In addition, through the setting of the timing belt 223, the sub-drive elements 22 on the same timing belt 223 can rotate at the same speed and rotation direction, which is beneficial for image acquisition and subsequent image recognition. Therefore, the details of the drive element 20 can be understood.

[0083] Regarding the image capturing element 30, please refer again. Figure 1BIn one embodiment, the imaging element 30 may be positioned adjacent to one side of the shelf assembly 10. In one embodiment, the relative position between the imaging element 30 and the shelf assembly 10 may be configured such that when the first drive member 21 rotates the shelf assembly 10, the front of one of the support frames 11 of the shelf assembly 10 faces the imaging element 30, and is not limited thereto. In one embodiment, when the front of one of the support frames 11 of the shelf assembly 10 faces the imaging element 30, an imaging range Ri of the imaging element 30 may cover all containers 2 on at least one shelf 13 of that support frame 11, and is not limited thereto. In one embodiment, the imaging subsystem 3 may include at least one imaging element 30, wherein when the front of one of the support frames 11 of the shelf assembly 10 faces the imaging element 30, the imaging range Ri of the imaging element 30 may cover all containers 2 on multiple shelves 13. In another embodiment, the imaging subsystem 3 includes multiple imaging elements 30, wherein the imaging range Ri of each imaging element 30 may correspond to different multiple shelves 13, and is not limited thereto. In one embodiment, the N-sided structure formed by the support frames 11 is rotatable relative to the base 21. When each support frame 11 rotates to a specific position, such as facing the image-capturing element 30, the shortest distance between each support frame 11 and the image-capturing element 30 in the horizontal direction (e.g., but not limited to the X or Y direction) is equal. Therefore, it is not necessary to readjust the relative position between the image-capturing element 30 and the frame assembly 10 each time the frame assembly 10 is rotated, which can shorten the image acquisition time and improve efficiency. In one embodiment, the image-capturing element 30 may be, for example, a camera, scanner, or other photosensitive element with image-capturing function, but is not limited to these. Therefore, the present invention does not require adjustment of the position of the image-capturing element 30 during use, providing convenience. Thus, the details of the image-capturing element 30 are already understood.

[0084] Next, details will be provided for image correction module 4, barcode recognition module 5, container feature recognition module 6, comparison module 7, and image segmentation module 70. Please refer to [link / reference] again. Figure 1AIn one embodiment, the image correction module 4, barcode recognition module 5, container feature recognition module 6, comparison module 7, and / or image segmentation module 70 may be, for example, functional modules. These modules can be implemented by at least one processor executing instructions stored in at least one computer program product on at least one non-transitory computer-readable medium, but are not limited thereto. In one embodiment, the processor for implementing these modules may be located in a computer, a mobile device, a cloud server, or other electronic device equipped with a processor. In one embodiment, the electronic device may have communication capabilities, enabling the image correction module 4, barcode recognition module 5, container feature recognition module 6, comparison module 7, and / or image segmentation module 70 to communicate with the image capturing element 30 or the database 8 via wired or wireless transmission, and are not limited thereto. In one embodiment, the image correction module 4, barcode recognition module 5, container feature recognition module 6, comparison module 7, and / or image segmentation module 70 may receive images from the image capturing element 30.

[0085] Furthermore, in one embodiment, the identification process of the container feature identification module 6 can be implemented, for example, by the processor executing a preset step flow. However, in another embodiment, the container feature identification module 6 can also be an artificial intelligence model, such as a trained machine learning model, which has the ability to automatically identify objects in an image, and is not limited thereto. In one embodiment, when the container feature identification module 6 is an artificial intelligence model, it can be set on a computer or a cloud server, and is not limited thereto.

[0086] Next, details of database 8 will be described. In one embodiment, database 8 may be stored in a computer, a mobile device, a cloud server, or other electronic device with storage, such as a hard drive, memory, cloud hard drive, etc., and is not limited thereto. In one embodiment, barcode identification module 5 and container feature identification module 6 may be electrically connected to or communicate with database 8, so that barcode identification module 5 and container feature identification module 6 can obtain data in database 8. In one embodiment, barcode database 81 may store multiple barcode data, wherein each barcode data may correspond to the identity information of a container 2 (e.g., a medicine bottle) or the information of the contents of container 2 (e.g., the information of the contents may be further converted into the identity information of container 2). For example, the barcode may be a one-dimensional barcode or a two-dimensional barcode, and the barcode may correspond to the information of the contents of container 2, thereby revealing the identity information of container 2, and is not limited thereto. In one embodiment, the frontal image database 82 may store data of frontal images of multiple containers 2, wherein the data of each frontal image may correspond to the identity information of a container 2. Here, "frontal image" is, for example, an image of the side of container 2 with a label, but is not limited thereto. In one embodiment, the color database 83 may store data of color distribution of multiple containers 2, wherein the data of each color distribution may correspond to the identity information of a container 2. Here, "color distribution" may be, for example, the distribution of the main colors on container 2 or the proportion of the main colors on container 2, but is not limited thereto. In one embodiment, the size database 84 may store data of the size of multiple containers 2, wherein the data of each size may correspond to the identity information of a container 2, but is not limited thereto. In one embodiment, the detail feature database 85 may store data of detail features of multiple containers 2, wherein the data of each detail feature may correspond to the identity information of a container 2. Here, "detail feature" may be, for example, a feature of a certain area on container 2, such as the details of text or labels in a certain area. It should be noted that the detail features of "text" or "label" here are mainly based on shape features, not on the content of the text or label, but are not limited thereto. It should be noted that the above "frontal view" refers to the entire container 2, such as the entire area covering the bottle head, neck, and body, while "detail features" refer to a specific area of ​​the container 2, such as a partial area of ​​the bottle body or a partial area of ​​the bottle head.

[0087] In addition, in one embodiment, when the container feature identification module 6 is an artificial intelligence model, the database 8 may also include an artificial intelligence model database 86 for storing various data required by the artificial intelligence model, such as data required during the training phase, various data established through training (e.g., various judgment logics), and / or data required during the actual use phase, and is not limited thereto.

[0088] It should be noted that the database types available in Database 8 can be added or removed as needed.

[0089] Further, in one embodiment, the barcode recognition module 5 can acquire an image from the imaging element 30 and compare the barcode on at least one container 2 in the image with data in the barcode database 81 to identify the identity information of the at least one container 2, and is not limited thereto. In one embodiment, the container feature recognition module 6 can acquire an image from the imaging element 30 and compare the frontal image of at least one container 2 in the image with data in the frontal image database 82 to identify the identity information of the at least one container 2, and is not limited thereto. In one embodiment, the container feature recognition module 6 can acquire an image from the imaging element 30 and compare the color distribution of at least one container 2 in the image with data in the color database 83 to identify the identity information of the at least one container 2, and is not limited thereto. In one embodiment, the container feature recognition module 6 can acquire an image from the imaging element 30 and compare the size of at least one container 2 in the image with data in the size database 84 to identify the identity information of the at least one container 2, and is not limited thereto. In one embodiment, the container feature recognition module 6 can acquire an image from the imaging element 30 and compare the detailed features of at least one container 2 in the image with data in the detailed feature database 85 to identify the identity information of the at least one container 2, and this is not limited to the above. Here, "comparison" can, for example, but not limited to, comparing the image acquired by the imaging element 30 with all data in a specific database, and the comparison results of the image and all data in the specific database can be converted into a numerical value, where a higher value indicates a closer match, thus finding the most suitable result. Furthermore, if the values ​​of all comparison results are below a preset threshold, it indicates that no matching result can be found, and this is not limited to the above.

[0090] Furthermore, in one embodiment, when the container feature identification module 6 is an artificial intelligence model, the container feature identification module 6 can automatically analyze the type of container in the image based on its machine learning results, and is not limited to this. In one embodiment, the container feature identification module 6 can be trained using a large number of images of various containers 2 during training, and after training is completed, as long as an image is input into the container feature identification module 6, the container feature identification module 6 can automatically determine the type of container 2 based on the judgment logic established during training, and is not limited to this.

[0091] Therefore, the details of the components of image recognition system 1 can be understood.

[0092] The image recognition system 1 of the present invention can be applied to the drug preparation process. Figure 3AA flowchart illustrating the operational process of a drug preparation device according to an embodiment of the present invention is shown, and please also refer to... Figures 1A to 2B .like Figure 3A As shown, first, step A1 is executed, where the hospital's in-hospital system transmits a medical order (e.g., a prescription) to the drug dispensing device, which receives the order. Next, step A2 is executed, where the operator of the drug dispensing device or the device itself (if automated) selects a suitable container 2 or syringe according to the medical order, where the container may contain a specific solution (e.g., a medication). Next, step A3 is executed, where the drug dispensing device (if automated) extracts the solution from container 2 using the syringe. Next, step A4 is executed, where the drug dispensing device itself (if automated) injects the solution into a solution bag using the syringe. Next, step A5 is executed, where the drug dispensing device itself (if automated) discards the syringe and container 2. In the above steps, image recognition system 1 can be used in step A2, for example, to identify the type of container 2 on the rack 10, thereby allowing the automated equipment to select the container 2 corresponding to the medical order or to provide container 2, and is not limited to this.

[0093] Figure 3B A flowchart illustrating the operational process of a drug preparation device according to another embodiment of the present invention is shown, and please also refer to... Figures 1A to 2B .like Figure 2B As shown, first, step B1 is executed: the hospital's internal system transmits a medical order (e.g., a prescription) to the drug dispensing device, which receives the order. Next, step B2 is executed: the operator of the drug dispensing device or the device itself (if automated) selects a suitable container 2 or syringe according to the medical order, where the container may contain a specific powder (e.g., a medication). Next, step B3 is executed: the drug dispensing device itself (if automated) dissolves the powder in the container into a solution. Next, step B4 is executed: the drug dispensing device itself (if automated) draws the dissolved solution from the container using the syringe. Next, step B5 is executed: the drug dispensing device itself (if automated) injects the dissolved solution into a solution bag using the syringe. Next, step B6 is executed: the drug dispensing device itself (if automated) discards the syringe and container 2. In the above steps, image recognition system 1 can be used in step B2, but is not limited to it. Furthermore, the frame assembly 10 and drive element 20 in the image acquisition subsystem 3 can also be used in step B3. For example, the drive element 20 can rotate the support frame 11, on which the container 2 required for step B3 is placed, toward the worker or other automated equipment, and allow the worker or other automated equipment to perform the contents of step B3 on the frame assembly 10, and is not limited thereto.

[0094] Furthermore, the frame assembly 10 of the present invention may also have a special design to facilitate use, for example, making it possible to Figure 3B Step B3 is executed more smoothly, and is not limited to this. Figure 4 A partially exploded view of an isolation door 40 according to an embodiment of the present invention is shown, and please also refer to... Figures 1A to 3B .

[0095] like Figure 4 As shown, the image acquisition subsystem 3 may also include an isolation door 40, located on one side adjacent to the frame assembly 10. For example, the frame assembly 10 may actually be located in a cavity 50 (shown in...). Figure 5 The cavity 50 is equipped with an isolation door 40. When the isolation door 40 is open, it connects the cavity 50 to the external space. When the isolation door 40 is closed, it closes the cavity 50. The isolation door 40 may include a body 41 and at least one window 42. The at least one window 42 may be provided with a first window panel 43A and a second window panel 43B. Each of the first window panel 43A and the second window panel 43B may be provided with a handle 44. The first window panel 43A and the second window panel 43B may be movable or slid in the X direction, for example, the first window panel 43A may move along the X direction, or the second window panel 43B may move in the opposite direction of the X direction. This allows a part of the at least one window 42 to be unobstructed by the first window panel 43A or the second window panel 43B, and exposes a part of the frame assembly 10, allowing personnel to access the frame assembly 10 through the unobstructed part of the window 42. Furthermore, the isolation door 40 may also include a lock 45 disposed on the body 41, wherein the lock 45 may be configured to open only when the maintenance rack assembly 10 is required or in response to an emergency, and is not limited thereto. In one embodiment, the first window panel 43A and the second window panel 43B may be made of transparent materials, such as glass, acrylic, or transparent conductive film (indium tinoxide, ITO), and are not limited thereto. In one embodiment, the first window panel 43A and the second window panel 43B may also be made of opaque materials.

[0096] Figure 5 A schematic diagram illustrating the use of an image-capturing subsystem 3 according to an embodiment of the present invention is shown, and please also refer to... Figures 1A to 4 .like Figure 5 As shown, the frame assembly 10 and the drive element 20 can be located in the cavity 50 ( Figure 5 In the cavity 50 (only part of the space is shown), the imaging element 30 may be located inside or outside the cavity 50, and a robotic arm 60 of other devices may be located inside the cavity 50.

[0097] In one embodiment, when it is necessary to place the clamping device 9 and the container 2 it clamps on the platform 12 of the frame assembly 10 (shown on...), Figure 1B and 1CWhen the container 2 is in use, the first window 43A or the second window 43B is moved to open a portion of the window 42, allowing part of the platform 12 on the support 10 to be exposed in the opened portion of the window 42. This allows the clamping device 9 and the container 2 it grips to be placed on the platform 12. Through the design of the isolation door 40, the first window 43A, and the second window 43B, only a small portion of the cavity 50 is opened at a time. Therefore, even if the contents of the container 2 contain volatiles, these volatiles are less likely to leak out of the cavity 50, thus improving safety, and this is not the only benefit.

[0098] In one embodiment, the image-capturing element 30 faces the isolation door 40; in other words, the image-capturing element 30 and the isolation door 40 are angled 180 degrees. When the image-capturing element 30 needs to capture an image of the container 2 on one of the support frames 11 of the frame assembly 10, the frame assembly 10 itself can rotate relative to the base 21, causing one of the support frames 11 to rotate 180 degrees from the isolation door 40 to face the image-capturing element 30. In one embodiment, the sub-drive element 22 can also rotate the stage 12 so that the container 2 can face the image-capturing element 30 at various angles to facilitate image capture. Furthermore, after the image-capturing element 30 captures an image, the acquired image can be transmitted to a device such as... Figure 1A The barcode recognition module 5, container feature recognition module 6, and comparison module 7 are used to identify container 2.

[0099] In this embodiment, when the robotic arm 60 needs to pick up a container 2 from one of the support frames 11 (e.g., according to a doctor's order), the frame assembly 10 itself can rotate so that one of the support frames 11 is oriented toward the robotic arm 60. At this time, an angle θ can be formed between the support frame 11 facing the robotic arm 60 and the isolation door 40. θ can be greater than or equal to 120 degrees and less than or equal to 160 degrees to reduce the impact of the robotic arm 60 on the imaging element 30. The robotic arm 60 can grip the groove 92 on the clamping device 9 with the container 2 (shown in...). Figure 2A Then, the clamping device 9 is removed from the platform 12, and is not limited thereto.

[0100] Therefore, the usage of the image acquisition subsystem 3 can be understood.

[0101] After the image acquisition element 30 acquires the image, the image recognition system 1 can execute an image recognition method to identify the container 2 in the image. Figure 6 The main flowchart of an image recognition method according to an embodiment of the present invention is shown, and please also refer to... Figures 1A to 5 .

[0102] like Figure 6As shown, first, step S1 is executed, where multiple containers 2 are placed on multiple stages 12. Next, step S2 is executed, where each sub-drive element 22 of the drive element 20 rotates each stage 12. Next, step S3 is executed, where the image-capturing element 30 acquires multiple images, each image corresponding to the multiple containers 2 at different angles. Next, step S4 is executed, where the image segmentation module 70 segments / or selects the multiple containers 2 from each image to form multiple container images. Next, step S5 is executed, where the container images of each container 2 are identified and compared. Finally, step S6 is executed, and the final result is output.

[0103] For steps S1 to S3 and S6, please refer to the description of the foregoing embodiments, and therefore will not be described in detail again. Regarding step S4, in one embodiment, the image segmentation module 70 can use various suitable methods to segment / or select multiple containers 2 in the image, such as binarization, clustering, histogram method, edge detection, region growing method, level set method, or wavelet transform method, etc., and is not limited thereto. In one embodiment, the multiple images obtained in step S3 may be, for example, images of the same set of containers 2 at different rotation angles. Since the position of each container 2 is still fixed in different images, when performing image segmentation / or selection in step S4, the image recognition system 1 can track the image after segmentation / or selection of each container 2 according to the position of each container 2 in the image, that is, the image recognition system 1 can know which multiple container images correspond to each container 2.

[0104] Details regarding step S5 Figure 7 A detailed flowchart of an image recognition method according to an embodiment of the present invention is shown, which is used to illustrate the details of step S5, and is used as an example for recognizing a single container image. Please also refer to... Figures 1A to 6 .exist Figure 7 In this context, the container feature identification module 6 is, for example, a trained artificial intelligence model, but is not limited to this.

[0105] like Figure 7As shown, first step S100 is executed, whereby the barcode recognition module 5 recognizes the barcode in the container image to obtain a first comparison result. The barcode recognition module 5 can identify whether there is a barcode in the container image. If there is, the recognized barcode is compared with the data in the barcode database 81 to generate a first comparison result. Furthermore, when the comparison is successful, the first comparison result may contain information related to the identity information of the container 2, such as, but not limited to, the name of the contents of the container (e.g., the name of a drug). When the comparison fails, the first comparison result may contain empty data. In addition, if the barcode recognition module 5 recognizes that there is no barcode in the container image, the barcode recognition module 5 may also output empty data as the first comparison result, and is not limited to this. Next, step S120 is executed, where the container feature recognition module 6 (a trained artificial intelligence model) identifies the appearance features of the container image to obtain a second comparison result. For example, the appearance features can be analyzed to identify the identity information of the container 2 and generate a second comparison result. When the identification is successful, the second comparison result may contain information related to the identity information of the container 2; when the identification fails, the second comparison result may contain empty data. Next, step S140 is executed, where the comparison module 7 compares the first comparison result and the second comparison result to generate a final result. When the first comparison result and the second comparison result match and neither is empty data, the final result may contain information related to the identity information of the container 2; when the first comparison result and the second comparison result do not match, the final result may contain information about an identification anomaly; and when both the first comparison result and the second comparison result are empty data, the final result may contain information about no container. Next, step S160 is executed, where the comparison module 7 outputs the final result.

[0106] In one embodiment, during the process of comparing the barcode in the container image with the data in the barcode database 81, the barcode recognition module 5 can compare the barcode in the container image with multiple data entries in the barcode database 81 and select the most matching data entry as the first comparison result, and is not limited thereto. Furthermore, the barcode recognition module 5 can preset a threshold; only when the similarity of the comparison is higher than or equal to the threshold will the data entry be used as the content of the first comparison result. In other words, if the similarity of all comparisons is lower than the threshold, then empty data is used as the content of the first comparison result, and is not limited thereto.

[0107] In one embodiment, since each stage 12 rotates, the same container 2 may have multiple images at different rotation angles. At some rotation angles, the container 2 may lack identifiable information, so the final result corresponding to the rotation angle may be identification of abnormal information or information of no container. At other rotation angles, the container 2 may have sufficient information, so the final result corresponding to the rotation angle may be information related to the identity information of the container 2. In this case, the electronic device 1 can use the information related to the identity information of the container 2 as the actual final result and output it, but is not limited to this.

[0108] Therefore, the details of image recognition methods can now be understood.

[0109] Figure 8 This is a detailed flowchart of an image recognition method according to another embodiment of the present invention, which illustrates the details of recognizing one of the container images in step S5. Please also refer to... Figures 1A to 7 .exist Figure 8 In this context, the container feature recognition module 6 is implemented, for example, by the processor executing a special image recognition algorithm. Furthermore, Figure 8 Details of some steps in the process are applicable. Figure 7 Therefore, the following mainly explains the differences between the needle segments.

[0110] like Figure 8 As shown, firstly, step S200 is executed, where the image correction module 4 corrects the container image. Next, step S220 is executed, where the barcode recognition module 5 identifies the barcode in the container image to obtain a first comparison result. The barcode recognition module 5 can identify whether the container image has a barcode; if so, it compares the identified barcode with the data in the barcode database 81 to generate a first comparison result. This step is applicable to… Figure 7 The description of step S100 is omitted here. Next, step S240 is executed. The container feature identification module 6 uses the frontal image of the container image as its appearance feature and identifies this appearance feature to obtain a second comparison result. The container feature identification module 6 can compare this appearance feature with data in the frontal image database 82 to identify the container's identity information and output the second comparison result. When the comparison is successful, the second comparison result may contain information related to the identity information of container 2; when the comparison fails, the second comparison result may contain empty data. Next, step S260 is executed. The comparison module 7 compares the first comparison result and the second comparison result to generate the final result. This step is similar to the description of step S140 and is therefore omitted here. Next, step S280 is executed, and the comparison module 7 outputs the final result.

[0111] In one embodiment, the image correction module 4 can be used to correct an image, wherein the image correction module 4 can determine whether the container image needs correction, and the situations in which the container image needs correction include, but are not limited to, image distortion, tilting, or blurring. The image correction module 4 can use various suitable image correction methods to correct the image, and the present invention is not limited thereto. In one embodiment, in Figure 7 Before starting the process, you can also execute the following steps first. Figure 8 The steps S200 (i.e., image correction) are not limited to this. Alternatively, in one embodiment, step S200 may be an optional step, and therefore step S200 may not be performed (i.e., image correction is not performed).

[0112] In one embodiment, during the process of the container feature identification module 6 comparing the appearance features of the container image with the data in the frontal image database 82, the container feature identification module 6 can compare the appearance features of the container image with multiple data entries and select the most matching data entry as the first comparison result, and is not limited thereto. Furthermore, the container feature identification module 6 can preset a threshold; only when the similarity of the comparison is higher than or equal to the threshold will that data entry be used as the content of the second comparison result. In other words, if the similarity of all data comparisons is lower than the threshold, then empty data is used as the content of the second comparison result, and is not limited thereto.

[0113] Therefore, the details of image recognition methods can now be understood.

[0114] Figure 9 A detailed flowchart of an image recognition method according to another embodiment of the present invention is shown, illustrating the details of recognizing one of the container images in step S5. Please also refer to... Figures 1A to 8 .exist Figure 9 In this context, the container feature recognition module 6 is implemented, for example, by the processor executing a special image recognition algorithm. Furthermore, Figure 9 Details of some steps in the process are applicable. Figure 7 and Figure 8 Therefore, the following mainly explains the differences between the needle segments.

[0115] like Figure 9 As shown, first step S300 is executed, where image correction module 4 corrects the container image. This step is applicable to... Figure 8The description of step S200 is omitted here. Next, step S310 is executed, whereby the barcode recognition module 5 recognizes the barcode in the container image to obtain a first comparison result. The barcode recognition module 5 checks whether the container image contains a barcode; if so, it compares the recognized barcode with data in the barcode database 81 to generate a first comparison result. Next, step S320 is executed, whereby the container feature recognition module 6 uses the color distribution of the container image (e.g., color configuration, proportion of main colors, etc.) as an appearance feature and recognizes this appearance feature. The container feature recognition module 6 can compare this appearance feature with data in the color database 83 to narrow down the recognition range. Next, step S330 is executed, whereby the container feature recognition module 6 uses the size of the container image (e.g., the height and width of container 2, or the height-to-width ratio) as an appearance feature and recognizes this appearance feature. The container feature recognition module 6 can compare this appearance feature with data in the size database 84 to further narrow down the recognition range. Next, step S340 is executed. The container feature recognition module 6 uses the detailed features of the container image (such as text or markings in a certain area) as appearance features and recognizes these appearance features. The container feature recognition module 6 can compare these appearance features with the data in the detailed feature database 85 to generate a second comparison result. When the comparison is successful, the second comparison result may contain information related to the identity information of the container 2, such as, but not limited to, the name of the contents of the container. When the comparison fails, the second comparison result may contain empty data. Next, step S350 is executed. The comparison module 7 compares the first comparison result and the second comparison result to generate a final result. Next, step S360 is executed. The comparison module 7 outputs the final result.

[0116] In one embodiment, the execution time of step S320 (i.e., the container feature recognition module 6 identifies the color distribution on the container image of container 2) can be earlier than the execution time of steps S330 (i.e., the container feature recognition module 6 identifies the size of the container image of container 2) and S340 (i.e., the container feature recognition module 6 identifies the text or logo on the container image of container 2). This ensures that the time for identifying the color distribution of container 2 is earlier than the time for identifying the size of container 2, or the time for identifying the color distribution of container 2 is earlier than the time for identifying the text or logo on container 2. Furthermore, the execution time of step S330 can be earlier than the execution time of step S340, ensuring that the time for identifying the size of container 2 is earlier than the time for identifying the text or logo on container 2. In another embodiment, the order of steps S320 to S340 is merely an example and can be adjusted according to requirements. The second comparison result can be output in the last step. Furthermore, steps S320 to S340 can also remove one or two steps as needed, or other steps can be added or replaced, for example, steps can be added or replaced. Figure 7Step S120 in the middle Figure 8 The present invention does not limit the steps S240 or other steps.

[0117] In one embodiment, during any of steps S320 to S340 executed by the container feature identification module 6, the container feature identification module 6 can compare the appearance features of the container image with multiple data entries and select the most matching data entry as the first comparison result, and is not limited thereto. Furthermore, the container feature identification module 6 can preset a threshold; only when the similarity of the comparison is higher than or equal to the threshold will that data entry be used as the output. In other words, if the similarity of all data comparisons is lower than the threshold, then empty data is output, and is not limited thereto.

[0118] Therefore, the details of image recognition methods can now be understood.

[0119] Therefore, the image recognition system 1, image recognition method, and image acquisition subsystem 3 of the present invention can improve ease of use. Alternatively, the image recognition system 1, image recognition method, and image acquisition subsystem 3 of the present invention can save a significant amount of time. Alternatively, the image recognition system 1, image recognition method, and image acquisition subsystem 3 of the present invention can improve security.

[0120] In one embodiment, the present invention can determine whether a product falls within the scope of protection of the present invention at least by examining the presence or absence of components, component configuration, mechanism observation, and / or operating mode of the product in dispute, and is not limited thereto. Alternatively, the present invention can also determine whether a product falls within the scope of protection of the present invention by examining the operating mode of the product in dispute, or by examining the algorithm of the product in dispute, and is not limited thereto. In one embodiment, the algorithm of the product in dispute can be obtained, for example, through reverse engineering, and is not limited thereto.

[0121] Details or features of the various embodiments of the present invention may be arbitrarily combined and used as long as they do not violate the spirit of the invention or conflict with it.

[0122] The above embodiments are merely illustrative examples for ease of explanation. The scope of the claims made in this invention should be determined by the claims of the patent application, and not limited to the above embodiments.

Claims

1. An image recognition system suitable for recognizing multiple containers, characterized in that, Include: An image-capturing element is used to acquire multiple images, wherein each image contains images of the multiple containers at different angles; A barcode identification module is used to obtain a first comparison result by identifying the barcodes on the multiple containers in the multiple images; A container feature recognition module is used to obtain a second comparison result by recognizing the appearance features of the multiple containers in the multiple images; and A comparison module is used to compare the first comparison result and the second comparison result to produce a final result.

2. The image recognition system according to claim 1, characterized in that, It also includes an image correction module for correcting the multiple images.

3. The image recognition system according to claim 1, characterized in that, The appearance features of the multiple containers include text or logos on the multiple containers.

4. The image recognition system according to claim 3, characterized in that, The appearance features of the multiple containers also include the size of the multiple containers, wherein the time point at which the container feature recognition module recognizes the size of the multiple containers is earlier than the time point at which the container feature recognition module recognizes the text or logos on the multiple containers.

5. The image recognition system according to claim 3, characterized in that, The appearance features of the multiple containers also include the color distribution of the multiple containers in the multiple images, wherein the time point at which the container feature recognition module recognizes the color distribution of the multiple containers is earlier than the time point at which the container feature recognition module recognizes the text or logo on the multiple containers.

6. An image recognition method applicable to recognizing multiple containers, the method being executed by an image recognition system, wherein the image recognition system includes an image capturing element, a barcode recognition module, a container feature recognition module, and a comparison module, characterized in that, The method also includes the following steps: The imaging element acquires multiple images, each of which contains images of the multiple containers at different angles. The barcode recognition module identifies the barcodes on the multiple containers in the multiple images to obtain a first comparison result. The container feature recognition module identifies the appearance features of multiple containers in the multiple images to obtain a second comparison result; and The comparison module compares the first and second comparison results to generate a final result.

7. The image recognition method according to claim 6, characterized in that, The image recognition system also includes an image correction module for correcting the multiple images.

8. The image recognition method according to claim 6, characterized in that, The appearance features of the multiple containers include text or logos on the multiple containers.

9. The image recognition method according to claim 8, characterized in that, The appearance features of the multiple containers also include the size of the multiple containers, wherein the time point at which the container feature recognition module recognizes the size of the multiple containers is earlier than the time point at which the container feature recognition module recognizes the text or logos on the multiple containers.

10. The image recognition method according to claim 8, characterized in that, The appearance features of the multiple containers also include the color distribution of the multiple containers in the multiple images, wherein the time point at which the container feature recognition module recognizes the color distribution of the multiple containers is earlier than the time point at which the container feature recognition module recognizes the text or logo on the multiple containers.

11. An image acquisition subsystem, characterized in that, Include: A frame assembly comprises multiple support frames arranged in an N-plane, where N is a positive integer greater than or equal to 4, and each of the multiple support frames has multiple platforms, each of the multiple platforms being used to hold a container. A drive element for rotating the plurality of platforms to rotate the container; as well as An image-capturing element for capturing an image of the container placed on each of the plurality of stages of one of the plurality of support frames.

12. The image acquisition subsystem according to claim 11, characterized in that, The N-plane of the plurality of support frames can be rotated so that the shortest distance between any of the plurality of support frames and the imaging element is equal.

13. The image acquisition subsystem according to claim 11, characterized in that, The drive element is used to rotate the frame assembly so that one of the plurality of support frames faces the imaging element.

Images