Apparatus for acquiring medicine inventory information, and method therefor

Abstract

According to the present disclosure, a method by which an electronic device acquires information is disclosed, the method comprising the steps of: acquiring an image; identifying one or more medicines included in the image; and acquiring, on the basis of the results of identifying the one or more medicines, inventory information for each medicine type included in the image.

Description

Device and method for acquiring pharmaceutical inventory information

[0001] The present disclosure relates to an apparatus and a method for acquiring inventory information of pharmaceuticals. More specifically, the present disclosure relates to an electronic apparatus and a method for identifying one or more pharmaceuticals included in an acquired target image and acquiring inventory information by type of pharmaceutical included in the target image based on the identification result.

[0002] In the case of pharmaceuticals, the need for thorough inventory management is particularly significant due to their nature, considering the potential for significant inconvenience to customers if necessary medications are not provided on time, the risk of health damage if incorrect medications are provided, and the possibility that certain medications may be used for undesirable activities such as misuse if they are incorrectly distributed.

[0003] Meanwhile, due to advancements in medical and pharmaceutical technologies, the types and quantities of medicines that pharmacies must store and manage are continuously increasing, requiring significant cost and effort to manage medicine inventory.

[0004] In this regard, prior art documents such as KR101674195B1 and KR102245345B1 may be referenced.

[0005] The problem that the present embodiment aims to solve is to provide an electronic device and a method thereof that acquire a target image, identify one or more pharmaceuticals included in the target image, and acquire inventory information by pharmaceutical type included in the target image based on the result of identifying one or more pharmaceuticals, in order to solve the aforementioned problem.

[0006] The technical problems to be solved by this embodiment are not limited to those described above, and other technical problems can be inferred from the following embodiments.

[0007] A method for obtaining information according to one embodiment includes the steps of: obtaining a target image; identifying one or more pharmaceuticals included in the target image; and obtaining inventory information by type of pharmaceutical included in the target image based on the result of identifying the one or more pharmaceuticals.

[0008] According to one embodiment, the step of identifying one or more pharmaceuticals may include: identifying one or more feature parts included in the target image; and obtaining type information of a pharmaceutical associated with a first feature part included in the one or more feature parts.

[0009] According to one embodiment, the step of obtaining type information of a pharmaceutical product associated with the first feature part may include the step of comparing the first feature part with one or more comparison images.

[0010] According to one embodiment, the step of comparing the first feature part with one or more comparison images includes the step of inputting the first feature part and the one or more comparison images into a pre-trained machine learning model; and the step of obtaining the output of the machine learning model, wherein the machine learning model may be pre-trained based on a plurality of drug images.

[0011] According to one embodiment, the step of comparing the first feature portion with one or more comparison images may include: determining one or more attributes associated with the first feature portion; and selecting one or more comparison images among a plurality of images included in a pharmaceutical image group based on the one or more attributes.

[0012] According to one embodiment, the step of determining one or more attributes may include the step of identifying logo information included in the first feature part.

[0013] According to one embodiment, the step of obtaining type information of a pharmaceutical product associated with the first feature part may include: a step of obtaining slope information associated with the first feature part; and a step of rotating the first feature part based on the slope information.

[0014] According to one embodiment, the step of obtaining type information of a drug associated with the first feature part includes the step of obtaining distortion information associated with the first feature part; and the step of correcting the first feature part based on the distortion information, and the step of correcting the first feature part may include the step of performing warping on the first feature part.

[0015] According to one embodiment, the method for obtaining the information further includes the step of obtaining shooting environment information of the target image, and the step of identifying one or more pharmaceuticals may include the step of converting the target image based on the shooting environment information.

[0016] According to one embodiment, the shooting environment information may include one or more of the following: product information of a camera for shooting the target image, setting information of the camera at the time of shooting the target image, distance information between the camera and the target at the time of shooting the target image, and angle information between the camera and the target at the time of shooting the target image.

[0017] According to one embodiment, the step of acquiring the inventory information may include: a step of determining the type of a plurality of pharmaceuticals existing in a space corresponding to the target image; and a step of acquiring location information of the plurality of pharmaceuticals.

[0018] According to one embodiment, the method for obtaining the information may further include the step of obtaining 3D modeling information corresponding to the target image based on the inventory information.

[0019] According to one embodiment, the step of acquiring the 3D modeling information may include: generating one or more geometric models corresponding to at least some of a plurality of pharmaceuticals existing in a space corresponding to the target image; and performing a texture mapping process on the one or more geometric models.

[0020] According to one embodiment, the method for obtaining the information further includes the step of obtaining storage environment information of one or more pharmaceuticals, and the electronic device can obtain the inventory information based on the storage environment information.

[0021] According to one embodiment, the storage environment information includes one or more attribute information related to a storage unit for storing one or more pharmaceuticals, and the one or more attribute information may include one or more of information for product identification of the storage unit, shape information of the storage unit, size information of the storage unit, and volume information of the storage unit.

[0022] According to one embodiment, the method for obtaining the information may further include the step of obtaining suggestion information related to the storage method of one or more pharmaceuticals based on the storage environment information and the inventory information.

[0023] According to one embodiment, the method for obtaining the information may further include the step of comparing the inventory information with previously stored computerized inventory information.

[0024] According to one embodiment, the method for obtaining the information may further include the step of providing information indicating the discrepancy when there is a discrepancy between the inventory information and the previously stored computerized inventory information.

[0025] An electronic device for acquiring information according to one embodiment includes a memory for storing instructions and a processor, wherein the processor is connected to the memory to acquire a target image, identifies one or more pharmaceuticals included in the target image, and acquires inventory information by type of pharmaceutical included in the target image based on the result of identifying the one or more pharmaceuticals.

[0026] Specific details of other embodiments are included in the detailed description and drawings.

[0027] According to the present disclosure, an apparatus and method for processing information can reduce costs and efforts for pharmaceutical inventory management.

[0028] The effects of the invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description in the claims.

[0029] FIG. 1 is a schematic diagram showing a system for managing inventory information according to one embodiment.

[0030] FIG. 2 is a diagram illustrating a method for an electronic device according to one embodiment to acquire inventory information of pharmaceuticals.

[0031] FIG. 3 is a drawing for explaining the external packaging of a pharmaceutical product according to one embodiment.

[0032] FIG. 4 is a drawing for exemplarily illustrating the main display surface of a pharmaceutical product according to one embodiment.

[0033] FIG. 5 is a drawing for exemplarily illustrating an information display surface of a pharmaceutical product according to one embodiment.

[0034] FIG. 6 is a drawing for exemplarily explaining shooting environment information of a target image according to one embodiment.

[0035] FIG. 7 is a drawing for exemplarily explaining a method of storing pharmaceuticals according to one embodiment.

[0036] FIG. 8 is a drawing for exemplarily illustrating a method of storing pharmaceuticals according to one embodiment.

[0037] FIG. 9 is a drawing for exemplarily explaining 3D modeling information according to one embodiment.

[0038] FIG. 10 is an example diagram of the configuration of an electronic device according to one embodiment.

[0039] The terms used in the embodiments have been selected to be as widely used as possible, taking into account their functions in the present disclosure; however, these may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section. Therefore, terms used in the present disclosure should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0040] When a part of a specification is described as “comprising” a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as “...part” or “...module” as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0041] The expression “at least one of a, b, and c” described throughout the specification may include ‘a alone’, ‘b alone’, ‘c alone’, ‘a and b’, ‘a and c’, ‘b and c’, or ‘a, b, and c all’.

[0042] The "terminal" mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, or laptop equipped with a web browser, and the portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all types of handheld-based wireless communication devices such as communication-based terminals like IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), and LTE (Long Term Evolution), smartphones, tablet PCs, etc.

[0043] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0045] In describing the embodiments, technical details that are well known in the technical field to which the present invention belongs and are not directly related to the present invention are omitted. This is intended to convey the essence of the present invention more clearly without obscuring it by omitting unnecessary explanations.

[0046] For the same reason, some components in the attached drawings have been exaggerated, omitted, or schematically depicted. Additionally, the size of each component does not entirely reflect its actual dimensions. Identical or corresponding components in each drawing have been assigned the same reference numbers.

[0047] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0048] At this time, it will be understood that each block of the process flow diagrams and combinations of the flow diagrams can be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create means to perform the functions described in the flow diagram block(s). Since these computer program instructions can also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory can also produce a manufactured item containing the means of instruction to perform the function described in the flow diagram block(s). Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in the flowchart block(s).

[0049] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). It should also be noted that in some alternative execution examples, the functions mentioned in the blocks may occur out of order. For instance, two blocks described in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to their corresponding functions.

[0050] FIG. 1 is a schematic diagram showing a system for managing inventory information according to one embodiment.

[0051] According to various embodiments, a system for managing inventory information includes an electronic device (110). A system for managing inventory information according to one embodiment may further include a manager's terminal (120). A system for managing inventory information according to one embodiment may further include a network that supports the transmission and reception of information between at least some of the electronic device (110), the manager's terminal (120), and external devices.

[0052] The electronic device (110) and the manager's terminal (120) of the system for managing inventory information may include memory and a processor, and may further include a transceiver depending on the embodiment. Additionally, each of the electronic device (110) and the manager's terminal (120) represents a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

[0053] Meanwhile, throughout the embodiments, the electronic device (110) and the administrator's terminal (120) are referred to as separate devices and servers, but they may be logically separated structures and may be implemented by functions separated from a single device or server. For example, depending on the embodiment, operations described as being performed by the administrator's terminal (120) may actually be performed by the electronic device (110), and in this case, the operation of transmitting and receiving information between the electronic device (110) and the administrator's terminal (120) may actually be understood as an operation of exchanging data within the electronic device (110).

[0054] According to one embodiment, the electronic device (110) and the administrator's terminal (120) may include a number of computer systems or computer software implemented as network servers. For example, at least some of the electronic device (110) and the administrator's terminal (120) may refer to computer systems and computer software that are connected to a sub-device capable of communicating with other network servers via a computer network such as an intranet or the Internet, receive requests for task execution, perform tasks thereon, and provide results. In addition, at least some of the electronic device (110) and the administrator's terminal (120) may be understood in a broader sense as including a series of applications capable of operating on a network server and various databases built internally. For example, at least some of the electronic device (110) and the administrator's terminal (120) may be implemented using network server programs provided in various ways depending on an operating system such as DOS, Windows, Linux, UNIX, or MacOS.

[0055] Meanwhile, for the sake of convenience of explanation, each operating entity has been referred to as an electronic device (110) and a manager's terminal (120), but these should be understood as a comprehensive type of device that corresponds to, includes, or can be included in, various types of devices such as computer devices and mobile communication terminals.

[0056] The electronic device (110) is a device that acquires and processes various information. Additionally, the electronic device (110) may provide various information. The electronic device (110) can perform various tasks to identify (e.g., acquire inventory information) and manage the inventory of pharmaceuticals.

[0057] More specifically regarding the operation of the electronic device (110), the electronic device (110) acquires a target image, identifies one or more pharmaceuticals included in the target image, and acquires inventory information by type of pharmaceutical included in the target image based on the result of identifying one or more pharmaceuticals. According to one embodiment, the electronic device (110) can identify one or more pharmaceuticals included in the target image by identifying one or more feature parts included in the target image and acquiring type information of pharmaceuticals associated with a first feature part included in the identified one or more feature parts.

[0058] According to one embodiment, the electronic device (110) can transmit the acquired inventory information to the manager's terminal (120).

[0059] The manager may correspond to an entity that intends to manage pharmaceutical inventory. The manager's terminal (120) may receive input from the manager, etc., or receive information from an electronic device (110), etc., and perform corresponding actions. According to one example, the manager's terminal (120) may acquire a target image and transmit it to the electronic device (110). According to another example, the manager's terminal (120) may receive inventory information by pharmaceutical type from the electronic device (110) and provide it to the manager through an output device such as a screen.

[0060] Meanwhile, the manager may include, but is not limited to, persons who sell pharmaceuticals to customers or persons who provide pharmaceuticals to sellers (e.g., vendors such as manufacturers).

[0061] More detailed information regarding the electronic device (110) and the manager's terminal (120) will be described later in FIGS. 2 to 9, etc.

[0062] Operations related to a series of information provision methods according to various embodiments may be implemented by a single physical device or may be implemented in a manner in which multiple physical devices are organically combined. For example, some of the components included in a system for managing inventory information may be implemented by one physical device, and the remaining parts may be implemented by another physical device. For example, one physical device may be implemented as part of an electronic device (110), and another physical device may be implemented as part of a manager's terminal (120) or other external device.

[0063] In some cases, each component included in the system for managing inventory information may be distributed and placed in different physical devices, and the distributed components may be organically combined to be implemented to perform the functions and operations of the system for managing inventory information. For example, the electronic device (110) of this specification may include at least one sub-device, and some operations described as being performed by the electronic device (110) may be performed by a first sub-unit, and other operations may be performed by a second sub-unit.

[0064] FIG. 2 is a diagram illustrating a method for an electronic device according to one embodiment to acquire inventory information of pharmaceuticals.

[0065] Referring to FIG. 2, an electronic device (110) according to one embodiment acquires a target image (210). According to one embodiment, the electronic device (110) may acquire the target image using its own shooting function. According to one embodiment, the electronic device (110) may acquire the target image using a directly connected shooting device. According to one embodiment, the electronic device (110) may receive the target image from another device (e.g., a manager's terminal (120)). In addition, various embodiments related to the operation of acquiring the target image may exist.

[0066] The electronic device (110) identifies one or more pharmaceuticals included in the target image (220). According to one embodiment, the electronic device (110) can identify one or more pharmaceuticals included in the target image by identifying one or more feature parts included in the target image and obtaining type information of the pharmaceutical associated with the first feature part included in the one or more feature parts. In this regard, one or more feature parts may include an object that can be identified, which may include, for example, logo information of the pharmaceutical.

[0067] Meanwhile, according to the embodiment, the electronic device (110) may not necessarily perform the "operation of obtaining type information of the pharmaceutical product associated with the first feature part" for all of one or more feature parts. For example, if the electronic device (110) identifies five feature parts in a target image that appear to be associated with one pharmaceutical product, and the associated pharmaceutical product is determined based on two of the feature parts, the operation of obtaining type information of the associated pharmaceutical product may not be performed for the remaining three feature parts.

[0068] For a more detailed understanding of one or more features, further reference may be made to FIG. 3 through 5. An example of an external packaging of a pharmaceutical product is illustrated in FIG. 3. Referring further to FIG. 3, the external packaging of a pharmaceutical product according to the example may be divided into a main display surface (310) and an information display surface (320). The main display surface (310) may include, for example, a front surface (311), a bottom surface (312), and a right surface (313), and the information display surface (320) may include, for example, a back surface (321), a left surface (322), and a top surface (323). Meanwhile, which surfaces the main display surface (310) and the information display surface (320) include are exemplary, and the example in FIG. 3 does not limit the scope of the present disclosure. Furthermore, wrapping a pharmaceutical product in such a cube-shaped package is merely an example for illustrative purposes, and it should be understood that, in addition to this type of packaging, the pharmaceutical product disclosed herein may be packaged in various forms and methods, such as in cylindrical containers, cellophane packaging, or PTP (Press-Through-Pack) packaging.

[0069] A more detailed example of the main display surface (310) is shown in reference numeral 410 of FIG. 4. Referring further to FIG. 4, the main display surface according to one example may include various information such as the name of the drug ("BB-Al" of Drawing No. 411), the manufacturer of the drug ("AA Company" of Drawing No. 412), the classification and quantity of the drug ("Over-the-counter drug" and "10 capsules" of Drawing No. 413), the efficacy of the drug ("Fast effect on allergies in liquid form" of Drawing No. 414 and "Antihistamine with reduced drowsiness" of Drawing No. 415), the ingredients and dosage of the drug ("CC salt 60 mg" of Drawing No. 416), information on symptoms for which the drug may be used ("House dust, pollen," "Sneezing," "Nasal allergy," "Runny nose, Nasal congestion" of Drawing No. 417), and information on the manufacturing method used for the drug ("DD method used" of Drawing No. 418).

[0070] At least some of the various information included in the main display surface such as this may be included in "one or more feature parts." For example, the electronic device (110) can identify information such as "BB-Al," "AA-Company," and "CC Salt 60mg" included in the target image, and obtain type information of the corresponding medicine based thereon.

[0071] A more detailed example of an information display surface (320) is shown in reference numeral 500 of FIG. 5. Referring further to FIG. 5, the information display surface according to the example may include various information such as active ingredients, efficacy, effect, usage, dosage, precautions for use, storage method, expiration date, and additives.

[0072] According to the embodiment, at least some of the various information included in such an information display surface may be further included in "one or more feature parts." For example, the electronic device (110) may identify information such as "active ingredients," "efficacy, effect," and "precautions for use" included in the target image, and further obtain type information of the corresponding medicine based thereon.

[0073] In particular, if there are images among the pharmaceutical images included in the target image where the main label is not visible or where it is difficult to identify the information on the main label, the information included in the information label can be actively identified and utilized to obtain type information of the pharmaceutical associated with the image.

[0074] Returning to FIG. 2, an electronic device (110) according to one embodiment can obtain information on the type of pharmaceutical associated with the first feature part by comparing the first feature part with one or more comparison images. For example, the electronic device (110) can associate the information on the type of pharmaceutical associated with the first feature part with the information on the type of pharmaceutical associated with the comparison image containing the text of "A Company" by comparing the first feature part containing the text of "A Company" with a comparison image containing the text of "B Company", a comparison image containing the text of "C Company", and a comparison image containing the text of "A Company", etc. In this regard, even if it is difficult to specify the type of pharmaceutical associated with the first feature part based on a single feature part (e.g., when there are multiple types of pharmaceuticals produced by "A Company"), the candidates for the type of pharmaceutical associated with the first feature part can be gradually narrowed down. Therefore, it can be understood that the electronic device (110) can specify the type of pharmaceutical associated with the first feature part by repeatedly performing this comparison operation on a plurality of feature parts.

[0075] The comparison image may include one or more images stored in advance, which do not necessarily have to be stored in the electronic device (110) (e.g., may be stored in an external database), but may be stored in a state that the electronic device (110) can retrieve as needed.

[0076] According to one embodiment, the electronic device (110) may obtain in advance the type of medicine that is handled or is likely to be handled at the place where the target image was taken (i.e., the place subject to inventory management), and further determine whether the image belongs to such a type, and select a comparison image for comparison with the first feature part. In other words, if the medicine is not handled at the place (e.g., a pharmacy) and is not likely to be handled, it is not subject to comparison with the feature part included in the target image, thereby increasing the accuracy of inventory identification and saving costs for inventory identification (e.g., resource consumption).

[0077] According to one embodiment, the electronic device (110) may determine one or more attributes associated with a first feature part and select one or more comparison images among a plurality of images included in a pharmaceutical image group based on the determined one or more attributes. As a related example, the electronic device (110) may determine what type of feature part the first feature part is and perform a comparison with a comparison image belonging to that type. For example, the electronic device (110) may determine that the first feature part is a feature part of the efficacy information type such as "fast effect on allergies in liquid form" and "antihistamine with reduced drowsiness" and perform a comparison with a comparison image of the same efficacy information type.

[0078] According to one example, the electronic device (110) can identify logo information included in the first feature part and determine one or more attributes associated with the first feature part by performing a comparison with a comparison image belonging to the logo based on the identified logo information. The logo information can be understood as information encompassing various types of logos, such as the logo of an importer, the logo of a distributor, and the logo of a manufacturer. In addition to the content of the text, the logo information may include various attribute information such as text color, text font, and background color other than the text, which will be described later, and furthermore, may include additional attribute information such as the shape or placement position of the logo, so it may be suitable for identifying a pharmaceutical product by comparison with a comparison image.

[0079] According to an embodiment, the electronic device (110) may determine more specific attributes beyond the type of feature part and perform a comparison with a comparison image having said attributes. For example, the electronic device (110) may determine that the text color of the first feature part is white, the text font is Gulim, and the background color other than the text is purple, and perform a comparison with a comparison image having text color, text font, and background color that match the above.

[0080] Meanwhile, the color contained in an image may vary depending on the shooting environment or various image processing methods (e.g., image format conversion for transmission between devices). As a method to account for such variations in image color, various embodiments may exist, such as excluding color from the attributes for filtering a comparison image, considering color but selecting a comparison image that has the corresponding attribute if it is within a certain similar range even if it does not exactly match, or selecting a comparison image by inferring how the color changed due to the shooting environment or image processing (e.g., obtaining the original color by inferring how the color changed and then applying that method in reverse).

[0081] Meanwhile, although "color" was used as an example for ease of understanding, it should be understood that for other attributes as well, it is not mandatory to select a comparison image of a "matching" attribute, and the above example regarding color may be applied to the action of selecting an appropriate comparison image.

[0082] In this way, instead of individually comparing the target image with all of the multiple images included in the drug image group, by performing an operation to select a comparison image to be compared based on one or more attributes associated with the first feature part, the number of times the operation to compare the target image with other images is repeated can be significantly reduced, thereby providing a technical effect of greatly reducing the cost (resources and time, etc.) required to identify one or more drugs included in the target image.

[0083] According to one embodiment, the electronic device (110) may perform Optical Character Recognition (OCR) on one or more of the target image and the comparison image and compare the two images based on the result. For example, even if the target image and the comparison image do not appear to be similar in color, font, material, etc., if the target image and the comparison image are common in that they both contain text such as "BB-R", "AA Company", "DD Method", etc., they may be judged to be the same type of pharmaceutical product.

[0084] According to one embodiment, the electronic device (110) can compare the first feature part and one or more comparison images by inputting the first feature part and one or more comparison images into a machine learning model that has been previously trained and obtaining the output of the machine learning model. It can be understood that the machine learning model has been previously trained based on a plurality of pharmaceutical images. Furthermore, based on the result of identifying one or more pharmaceuticals included in the target image according to the method of the present disclosure and the result of obtaining inventory information by type of pharmaceutical included in the target image, the electronic device (110) may further obtain one or more pharmaceutical images (which may be the target image itself or images obtained based on the target image), and may further train the machine learning model based on the one or more pharmaceutical images obtained.

[0085] According to one embodiment, the electronic device (110) may appropriately process or modify the first feature portion to increase the identifiability of the pharmaceutical product. According to one example, the electronic device (110) may acquire slope information associated with the first feature portion and rotate the first feature portion based on the acquired slope information. The slope information may include a two-dimensional slope or a three-dimensional slope.

[0086] According to another example, the electronic device (110) may acquire distortion information associated with the first feature and correct the first feature based on the acquired distortion information. The distortion associated with the first feature may include elements such as shape distortion, excessive gathering of light, and the formation of shadows, which may occur due to limitations of the inherent structure of the camera. As an example of an operation to correct the first feature, the electronic device (110) may perform warping on the first feature.

[0087] The aforementioned "operation of appropriately processing or modifying the first feature part" can be understood as an operation of determining, based on the image part itself, whether there is a slope or distortion for each feature part (i.e., determining whether there is a slope or obscuration by shadows, etc., based on the shape or color of the image itself).

[0088] Meanwhile, the electronic device (110) according to one embodiment may acquire shooting environment information of a target image and transform the target image based on the shooting environment information. By transforming the target image in this way, the identifiability of one or more pharmaceuticals and the accuracy of such identification can be increased. The operation of transforming the target image based on the shooting environment information can be understood as an operation of transforming the target image based on external environmental factors rather than the shape or color of the image itself (unlike the operation of appropriately processing or modifying the first feature part).

[0089] The shooting environment information may include one or more of the following: product information of the camera for shooting the target image, setting information of the camera at the time of shooting the target image, distance information between the camera and the target at the time of shooting the target image, and angle information between the camera and the target at the time of shooting the target image. Based on one or more of the product information of the camera and the setting information of the camera, the electronic device (110) can determine the type and degree of distortion that may be caused by the camera and can appropriately transform the target image so that the distortion is corrected. Similarly, based on one or more of the distance information between the camera and the target at the time of shooting the target image and the angle information between the camera and the target at the time of shooting the target image, the electronic device (110) can determine the degree of tilt of the pharmaceutical image included in the target image and the size of the image, and can appropriately transform the target image based thereon.

[0090] Meanwhile, the operation of "transforming the target image" described above may include an operation of "transforming some components included in the target image," and the method of transformation may differ for each component. For example, when a camera photographs multiple pharmaceuticals, since there may be differences in the degree of tilt or distance from the camera for each pharmaceutical, transformation according to different methods may be performed for each pharmaceutical.

[0091] The electronic device (110) can compare the target image (or the component included in the target image) converted in this way with the comparison image described above.

[0092] According to an embodiment, the electronic device (110) may acquire shooting environment information of a target image and convert a comparison image based on the shooting environment information. It can be expected that the comparison image has a higher quality than the target image or has a lower degree of tilt or distortion. Since leaving the target image as is can reduce the complexity of the procedure when comparing the target image with a number of comparison images, converting the comparison image in this way can increase the identifiability of one or more pharmaceuticals and the accuracy of such identification. Meanwhile, when converting the comparison image, it can be understood that the conversion of the comparison image is achieved by applying the method of “converting the target image” described above in reverse (for example, if the target image was to be rotated 20 degrees clockwise, the comparison image is instead rotated 20 degrees counterclockwise), but the method of the present disclosure is not necessarily limited thereto.

[0093] The electronic device (110) may also compare the target image (or the component included in the target image) with a comparison image converted in this way.

[0094] By converting the target image or the comparison image, a technical effect can be achieved in which the influence of various interfering factors (e.g., tilting, distortion, enlargement or reduction of size, etc.) that may affect the identification of the pharmaceutical product contained in the target image is minimized.

[0095] Various embodiments may exist regarding the specific method by which the electronic device (110) obtains shooting environment information of a target image. According to one embodiment, the electronic device (110) may receive shooting environment information. According to one embodiment, the electronic device (110) may receive information that forms the basis of the shooting environment information. For example, the electronic device (110) may receive camera position information and obtain shooting environment information by calculating the distance and angle to the shooting targets based thereon.

[0096] According to an embodiment, the electronic device (110) may generate shooting environment information by utilizing information already acquired in relation to some of the shooting targets. For example, if the electronic device (110) has accurately identified the size and shape of some of the pharmaceuticals among the shooting targets, it may determine how far the pharmaceutical is from the camera and what the angle is by comparing the accurately identified size and shape with the size and shape of the pharmaceuticals captured by the camera. In other words, it can be understood that the electronic device (110) can acquire shooting environment information based on the shooting targets for which it has already accurately identified related information, and subsequently use the shooting environment information to transform the target image (or at least some of the shooting targets included in the target image). (In other words, the shooting target image may be transformed based on the shooting environment information, and the shooting environment information may be acquired based on the shooting target image. That is, information acquisition may be performed in both directions.)

[0097] Shooting environment information of a target image is illustrated exemplarily in FIG. 6. Referring further to FIG. 6 (600), the shooting environment information according to one embodiment may include one or more of product information of the camera (601) and setting information of the camera (601).

[0098] Additionally, the shooting environment information according to one embodiment may include distance information (605) between the camera (601) and the shooting target (602). In this regard, the distance information (605) according to one embodiment may include the length of a line (604) connecting the camera (601) and the shooting target (602), but the scope of the present disclosure is not necessarily limited thereto.

[0099] In addition, the shooting environment information according to one embodiment may include angle information (606) between the camera (601) and the shooting target (602). In this regard, the angle information (606) according to one embodiment may include the angle formed between a reference line (603) in the direction the camera (601) gazes at directly and a line (604) connecting the camera (601) and the shooting target (602), but the scope of the present disclosure is not necessarily limited thereto.

[0100] Returning to FIG. 2, the electronic device (110) obtains inventory information by type of medicine included in the target image based on the result of identifying one or more medicines (230). The medicines for which the electronic device (110) determines the type may include medicines that are directly exposed in the target image, but are not necessarily limited thereto. For example, the electronic device (110) according to one embodiment may determine the types of multiple medicines existing in the space corresponding to the target image. The multiple medicines "existing in the space" corresponding to the target image can be understood to include medicines for which at least a part thereof is directly exposed in the target image, as well as medicines for which at least a part thereof is not directly exposed but can be presumed to exist in a hidden place.

[0101] Meanwhile, regarding specific methods for presuming existence in a hidden place, examples may exist in which the amount of additional pharmaceuticals that can be stored in an internal space is estimated based on length information, such as the depth of the place where the pharmaceuticals are stored (shelf or medicine cabinet, etc.), or internal volume information. However, the scope of the present disclosure is not limited to such examples.

[0102] The electronic device (110) can obtain location information of a pharmaceutical product whose type has been determined. According to one embodiment, the electronic device (110) can obtain location information of a pharmaceutical product based on one or more of the relative positional relationship with the place where the pharmaceutical product is stored and the relative positional relationship with other pharmaceutical products. For example, if the shelf for storing pharmaceutical products has a physical structure that can be divided into several compartments, the electronic device (110) can name each compartment of the shelf as the first compartment, the second compartment, the nth compartment, etc., and obtain location information of the pharmaceutical product as "inside the first compartment," "inside the second compartment," "inside the nth compartment." In another example, the electronic device (110) can determine the relative positions among the drugs and set the x-coordinate of the drug located furthest to the left to 1, and increase the x-coordinates one by one in order of relative to the drug located to the right, and likewise set the y-coordinate of the drug located furthest to the bottom to 1, and increase the y-coordinates one by one in order of relative to the drug located to the top. In addition, there may be various other examples of setting the position information of the drugs, and no specific example limits the scope of the present disclosure.

[0103] In FIG. 7, examples of how pharmaceuticals are stored are described. Referring further to FIG. 7 (700), the outer packaging of each pharmaceutical may include information such as its name and dosage. As shown in the example in FIG. 7, "AAA tablet 10 / 40 milligrams" and "AAA tablet 5 / 20 milligrams" exist respectively, even pharmaceuticals with the same name may differ in quantity or dosage depending on the packaging unit, and this can be identified by the outer packaging. In addition, as can be seen in the example in FIG. 7 that the packaging of "AAA tablet 10 / 40 milligrams" and the packaging of "AAA tablet 5 / 20 milligrams" have different colors, it may be identified that the packaging units have different quantities or dosages based on attributes such as the color or shape of the outer packaging.

[0104] Meanwhile, although expressed as "different quantities or dosages" for the sake of convenience of understanding, there may be various other embodiments of pharmaceuticals with the same name, such as having different efficacies, using different manufacturing methods, or having different symptoms for use, and the above description may be applied to such embodiments.

[0105] In addition, while AAA tablets and BBB tablets are each independently packaged in cube-shaped containers, there may be various embodiments regarding specific packaging methods, such as multiple drugs from the same manufacturer being stored together in a large group like "K Pharmaceutical" in the upper left corner, or being packaged in cylindrical containers like "CCC" in the center left corner.

[0106] Meanwhile, regarding packaging methods, according to one example, information to identify the drug may be written by hand on the outer packaging of the drug. Referring further to FIG. 8 (800), it can be seen that information such as “XX Feed (Company D),” “XX Feed,” or “XX Feed Tablet” is written by hand. In cases where it is written by hand in this manner, the method described above, “performing OCR and comparing the target image and the comparison image based on the result,” may be particularly useful. In this regard, since the information written by hand is not information included in the uniformly printed packaging, it can be understood that judging and comparing the actual content may be more effective in determining the type of drug.

[0107] Returning to FIG. 2, the electronic device (110) according to one embodiment can acquire 3D modeling information corresponding to a target image based on the acquired inventory information. By acquiring 3D modeling information in this way, inventory management can be performed by modifying the inventory information when the inflow or outflow of pharmaceuticals is detected. Compared to the method of continuously checking inventory directly, the method of detecting the inflow or outflow of pharmaceuticals can be more accurate in terms of cost and accuracy. Furthermore, while a relatively high-performance camera (e.g., capable of generating high-resolution images) is required for checking inventory directly, the task of detecting the inflow or outflow of pharmaceuticals can be performed by a relatively low-performance camera; therefore, the method of detecting the inflow or outflow of pharmaceuticals can be understood as having higher accessibility (especially for small business owners, etc.).

[0108] According to a more specific embodiment, the electronic device (110) can obtain 3D modeling information by generating one or more geometric models corresponding to at least some of a plurality of pharmaceuticals existing in a space corresponding to a target image, and by performing a texture mapping process on the generated one or more geometric models. As an example of the operation of generating a geometric model, the electronic device (110) may model the outer packaging of the pharmaceutical as a cube-shaped mesh having dimensions on the x, y, and z axes, or as a cylinder-shaped mesh having dimensions of radius r and height l. A pharmaceutical type (e.g., code) may be mapped to the geometric model.

[0109] The texture mapping process may include a process of applying textures, such as images and patterns, to a 3D model to impart various attributes such as color, texture, and detail. As a specific method of the texture mapping process, the electronic device (110) may perform the texture mapping process by performing an unwrapping process of unfolding each part of the 3D model and placing it in a 2D space, a process of creating a texture (e.g., representing color, light reflection, transparency, brightness, etc.) by generating an image using graphic software or using a previously generated image, and a mapping process of applying the texture to each part of the 3D model by mapping the texture onto the unwrapped map. However, this is merely a specific example to aid in understanding the texture mapping process, and the scope of the present disclosure is not limited to such an example.

[0110] In the present disclosure, an example of determining x-coordinates and y-coordinates was described as an example of obtaining pharmaceutical location information, and an example of a cube having dimensions on the x-axis, y-axis, and z-axis in 3D modeling was described. However, in the case of these examples, coordinates or axes were directly mentioned for the convenience of explanation, and examples in which such information is actually defined or expressed as a matrix or vector should also be understood to fall within the scope of the present disclosure.

[0111] According to one embodiment, the electronic device (110) can acquire storage environment information for one or more pharmaceuticals, and the electronic device (110) can acquire inventory information based on the acquired storage environment information. The storage environment information may include one or more attribute information related to a storage unit for storing one or more pharmaceuticals. One or more attribute information according to one example may include one or more of information for product identification of the storage unit, shape information of the storage unit, size information of the storage unit, and volume information of the storage unit. The information for product identification of the storage unit may include, for example, information such as the manufacturer and product name of the storage unit (i.e., information such as the manufacturer and product name of the shelf).

[0112] Based on one or more of the shape information, size information, and volume information of the storage unit, the electronic device (110) can determine the size and shape of a three-dimensional space in which items such as medicines can be stored in the storage unit. The size and shape of the three-dimensional space thus determined can be used to obtain the aforementioned 3D modeling information. Alternatively, based on the size and shape of the three-dimensional space thus determined, the electronic device (110) can estimate how much more medicine can be stored in the internal space of the place where medicines are stored (related to the operation of determining the type of multiple medicines “existing in space” corresponding to the aforementioned target image).

[0113] Meanwhile, in addition to the shape information, size information, and volume information of the storage unit described above, various attribute information may be included in the storage environment information of the present disclosure, such as attribute information regarding how well the storage unit maintains its shape, attribute information regarding whether the storage unit is fixed to a wall (or whether it can move inside the wall, etc.), attribute information regarding whether the storage unit is fixed to a partition wall, attribute information regarding whether the storage unit has a fixed positional relationship with other shelves, and attribute information regarding whether the storage unit is easy to dismantle.

[0114] According to one embodiment, the electronic device (110) can obtain suggestion information regarding the storage method of one or more pharmaceuticals based on storage environment information and inventory information. For example, the electronic device (110) can suggest a change in the storage location and loading method so that pharmaceuticals that are frequently brought in and taken out can be stored in a location within easy reach of people.

[0115] According to one embodiment, the electronic device (110) can compare the inventory information obtained through the operation of reference numeral 230 with the previously stored computerized inventory information. If there is a discrepancy between the inventory information obtained through the operation of reference numeral 230 and the previously stored computerized inventory information, the electronic device (110) may provide information indicating the discrepancy. The information indicating the discrepancy may be provided to the administrator's terminal (120), but the recipient of such information is not limited to the administrator's terminal (120). Specific examples of cases where a discrepancy exists include cases where an item that does not exist in the computerized inventory is identified on the target image, and cases where the quantity of item A in the computerized inventory differs from the quantity of item A on the target image, and specific examples do not limit the scope of the present disclosure.

[0116] In this way, by comparing the inventory information with the previously stored computerized inventory information, the electronic device (110) can verify whether the acquired inventory information is accurate and whether the previously stored computerized inventory information is accurate.

[0117] Meanwhile, according to the embodiment (especially in environments where it is difficult to use a high-performance camera), the electronic device (110) can more easily estimate the type of medicine included in the target image by referring to the computerized inventory and more easily determine whether there is a change in inventory. As previously explained regarding the "operation of acquiring 3D modeling information," while a relatively high-performance camera (e.g., capable of generating high-resolution images) is required for direct identification of inventory, tasks such as detecting the inflow or outflow of medicine may be performed by a relatively low-performance camera. Therefore, the method of more easily determining whether there is a change in inventory by referring to the computerized inventory can be understood as having higher accessibility.

[0118] FIG. 9 is a drawing for exemplarily explaining 3D modeling information according to one embodiment.

[0119] Referring to the example of FIG. 9 (900), the 3D modeling information according to one embodiment may include a plurality of geometric models that reflect the shape of each outer packaging, and a pharmaceutical type may be mapped to at least some of the plurality of geometric models. Furthermore, a texture may be mapped to at least some of the plurality of geometric models.

[0120] 3D modeling information according to one embodiment may reflect where each geometric model is stored on the storage unit. Furthermore, 3D modeling information according to one embodiment may reflect one or more attribute information related to the storage unit. According to an embodiment, the 3D modeling information may help facilitate the task of storing additional pharmaceuticals by reflecting information about empty spaces in the storage unit where pharmaceuticals are not yet stored.

[0121] 3D modeling information according to the example (900) of FIG. 9 may be represented as an image reflecting various information, but 3D modeling information does not necessarily have to be represented as an image. For example, 3D modeling information may include one or more pieces of information represented as text, such as {Medicine 1: coordinates (1,0,3), texture (red, no reflection, no transparency, dark contrast)}. In addition, there may be various forms for representing 3D modeling information, and no specific form limits the scope of the present disclosure.

[0122] FIG. 10 is an example diagram of the configuration of an electronic device according to one embodiment.

[0123] Referring to FIG. 10, the electronic device (110) includes a processor (1020) and a memory (1030), and may further include a transceiver (1010) according to the embodiment. The electronic device (110) can be connected to a manager's terminal (120) and other external devices, etc., through the transceiver (1010) and exchange data.

[0124] The processor (1020) may include at least one device described through FIGS. 1 to 9 or perform at least one method described through FIGS. 1 to 9. The memory (1030) may store information for performing at least one method described through FIGS. 1 to 9. The memory (1030) may be a volatile memory or a non-volatile memory.

[0125] The processor (1020) can control an electronic device (110) for executing a program and providing information. The code of the program executed by the processor (1020) can be stored in memory (1030).

[0126] In addition, the electronic device (110) of one embodiment may further include an interface capable of providing information to the user.

[0127] Meanwhile, the present specification and drawings disclose preferred embodiments of the present invention. Although specific terms have been used, they are used merely in a general sense to facilitate the explanation of the technical content of the present invention and to aid in understanding the invention, and are not intended to limit the scope of the present invention. It is obvious to those skilled in the art that, in addition to the embodiments disclosed herein, other variations based on the technical concept of the present invention are possible.

[0128] The recommendation server, electronic device, or terminal according to the embodiments described above may include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, and user interface devices such as a touch panel, key, button, etc. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable code or program instructions executable on the processor. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD (Digital Versatile Disc)). Computer-readable recording media may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The medium may be readable by a computer, stored in memory, and executed by a processor.

[0129] The present embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the embodiment may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., capable of executing various functions by the control of one or more microprocessors or other control devices. Similar to how components may be implemented as software programming or software elements, the present embodiment may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, Python, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Additionally, the present embodiment may employ prior art for electronic configuration, signal processing, and / or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations. The above terms may include the meaning of a series of software processes (routines) in conjunction with processors, etc.

[0130] The aforementioned embodiments are merely examples, and other embodiments may be implemented within the scope of the claims set forth below.

Claims

1. In a method for obtaining information from an electronic device, Step of acquiring a target image; A step of identifying one or more pharmaceuticals included in the above target image; and A method for obtaining information, comprising the step of obtaining inventory information by type of medicine included in the target image based on the result of identifying one or more medicines.

2. In Paragraph 1, The step of identifying one or more of the above-mentioned drugs A step of identifying one or more feature parts included in the above target image; and A method for obtaining information, characterized by including the step of obtaining type information of a pharmaceutical product associated with a first feature part included in one or more of the above feature parts.

3. In Paragraph 2, The step of obtaining type information of a drug associated with the above-mentioned first feature part A method for obtaining information, characterized by including the step of comparing the first feature portion with one or more comparison images.

4. In Paragraph 3, The step of comparing the above-mentioned first feature part with one or more comparison images A step of inputting the first feature part and the one or more comparison images into a pre-trained machine learning model; and It includes the step of obtaining the output of the machine learning model above, and A method for acquiring information, characterized in that the machine learning model is trained based on a plurality of drug images.

5. In Paragraph 3, The step of comparing the above-mentioned first feature part with one or more comparison images A step of determining one or more attributes associated with the first feature part; A method for obtaining information characterized by including the step of selecting one or more comparison images among a plurality of images included in a pharmaceutical image group based on one or more of the above attributes.

6. In Paragraph 5, The step of determining one or more of the above attributes A method for obtaining information, characterized by including a step of identifying logo information included in the first feature part.

7. In Paragraph 2, The step of obtaining type information of a drug associated with the above-mentioned first feature part A step of obtaining slope information associated with the first feature part; and A method for obtaining information, characterized by including the step of rotating the first feature part based on the above-mentioned slope information.

8. In Paragraph 2, The step of obtaining type information of a drug associated with the above-mentioned first feature part A step of obtaining distortion information associated with the first feature part; Based on the distortion information above, the method includes the step of correcting the first feature part, and The step of correcting the first feature part above A method for acquiring information, characterized by including the step of performing warping on the first feature portion.

9. In Paragraph 1, The method further includes the step of obtaining shooting environment information of the above target image, and The step of identifying one or more of the above-mentioned drugs A method for acquiring information, characterized by including a step of converting the target image based on the above-mentioned shooting environment information.

10. In Paragraph 9, The above shooting environment information Product information of the camera for capturing the above target image, The camera's setting information at the time of capturing the above target image, Distance information between the camera and the subject at the time of capturing the subject image, and Angle information between the camera and the subject at the time of capturing the above target image A method for obtaining information characterized by including one or more of the following.

11. In Paragraph 1, The step of obtaining the above inventory information is A step of determining the types of multiple pharmaceuticals existing in a space corresponding to the above target image; and A method for obtaining information characterized by including the step of obtaining location information of the plurality of pharmaceuticals.

12. In Paragraph 1, A method for acquiring information, characterized by further including the step of acquiring 3D modeling information corresponding to the target image based on the above inventory information.

13. In Paragraph 12, The step of acquiring the above 3D modeling information A step of generating one or more geometric models corresponding to at least some of the plurality of pharmaceuticals existing in a space corresponding to the above target image; and A method for acquiring information, characterized by including the step of performing a texture mapping process on one or more geometric models.

14. In Paragraph 1, The method further includes the step of obtaining storage environment information for one or more of the above-mentioned pharmaceuticals, and A method for acquiring information, characterized in that the electronic device acquires the inventory information based on the storage environment information.

15. In Paragraph 14, The above storage environment information It includes one or more attribute information related to a storage unit for storing one or more of the above-mentioned pharmaceuticals, and The above one or more attribute information Information for product identification in the above storage unit, Shape information of the above storage unit, Size information of the above storage section, and Volume information of the above storage unit A method for obtaining information characterized by including one or more of the following.

16. In Paragraph 14, A method for obtaining information, characterized by further including the step of obtaining suggestion information regarding a storage method of one or more pharmaceuticals based on the storage environment information and the inventory information.

17. In Paragraph 1, A method for obtaining information, characterized by further including the step of comparing the above-mentioned inventory information with previously stored computerized inventory information.

18. In Paragraph 17, A method for obtaining information, characterized by further including the step of providing information indicating the discrepancy when there is a discrepancy between the above-mentioned inventory information and the above-mentioned previously stored computerized inventory information.

19. A computer-readable, non-transient recording medium having a program for executing the method of paragraph 1 on a computer.

20. In an electronic device for acquiring information, It includes memory and a processor for storing instructions, The above processor is connected to the above memory, Acquire the target image, and Identify one or more pharmaceuticals included in the above target image, and An electronic device that obtains inventory information by type of medicine included in the target image based on the result of identifying one or more medicines.

Images