Electronic device and pill identification method thereof

Abstract

A method by which at least one processor identifies a pill, of the present disclosure, may comprise the steps of: receiving, through a camera, an image including at least one pill; displaying the image on a display; identifying the at least one pill in the image; and overlaying at least one graphic object on the at least one pill.

Description

How to identify electronic devices and their pills

[0001] The present disclosure relates to an electronic device and a method for identifying a pill thereof, and specifically, to a technique for identifying a pill on an image in which the pill is captured.

[0002] Regarding medications, it is crucial that pharmacists prescribe a fixed number of pills and that patients take the prescribed amount. When the number of pills is very large, accurately counting them and classifying them into the prescribed quantity can be time-consuming. When using electronic devices to identify pills, it is important for the user to be aware of the number and location of the pills in real time.

[0003] According to one embodiment of the present disclosure, the technical problem is to identify a pill included in an image and display it in a manner that is easy for the user to recognize.

[0004] According to one embodiment of the present disclosure, the technical problem is to enable a user to easily recognize and classify pills by identifying and displaying pills included in an image in real time.

[0005] The present disclosure may be implemented in various ways, including a method, an apparatus (system) and / or a computer program stored on a computer-readable storage medium, and a computer-readable storage medium on which the computer program is stored.

[0006] A method for identifying a pill performed by at least one processor according to one embodiment of the present disclosure may include: receiving an image containing at least one pill through a camera; displaying the image through a display; identifying the at least one pill on the image; and overlaying at least one graphic object on the at least one pill.

[0007] According to one embodiment, the step of identifying the at least one pill may include: identifying the number and location of the at least one pill; and identifying at least one of the size and color of the at least one pill.

[0008] According to one embodiment, the size of the at least one graphic object can be determined based on the size of the at least one pill.

[0009] According to one embodiment, the size of the at least one graphic object can be determined based on the width information and height information of the at least one pill.

[0010] According to one embodiment, the color of the at least one graphic object can be determined based on the color of the at least one pill.

[0011] A method for identifying the pill according to one embodiment may further include: a step of confirming the color value of the center point of the at least one pill; a step of determining a representative color value based on the color value of the center point of the at least one pill; a step of determining a complementary color value of the representative color value; and a step of determining a color with maximized saturation of the complementary color value as the color of the at least one graphic object.

[0012] A method for identifying the pill according to one embodiment may further include: a step of confirming the color value of the center point of the at least one pill; a step of determining a representative color value based on the color value of the center point of the at least one pill; a step of determining a complementary color value of the representative color value; and a step of determining one color among a predetermined plurality of colors that is most similar to the color value of the complementary color value as the color of the at least one graphic object.

[0013] According to one embodiment, the at least one graphic object and the at least one pill can correspond one-to-one.

[0014] A method for identifying the pill according to one embodiment may further include the step of changing the size of the at least one graphic object based on a first user input; and the step of changing the color of the at least one graphic object based on a second user input.

[0015] In a method for identifying the pill according to one embodiment, the shape of the at least one graphic object may be one of a circle, an ellipse, and a polygon.

[0016] An electronic device according to one embodiment of the present disclosure comprises a camera; a display; and a processor, wherein the processor may be configured to receive an image including at least one pill through the camera, display the image through the display, identify the at least one pill on the image, and overlay at least one graphic object on the at least one pill.

[0017] According to one embodiment of the present disclosure, pills included in an image can be identified and displayed in a manner that is easy for the user to recognize.

[0018] According to one embodiment of the present disclosure, by identifying and displaying pills included in an image in real time, a user can easily recognize and classify the pills.

[0019] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims.

[0020] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto.

[0021] FIG. 1 is a schematic diagram illustrating an example of an electronic device according to one embodiment of the present disclosure.

[0022] FIG. 2 is a schematic diagram showing a configuration in which an information processing system and a plurality of user terminals are connected to enable communication between them according to one embodiment of the present disclosure.

[0023] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0024] FIG. 4 is a flowchart of the operation of an electronic device according to one embodiment of the present disclosure.

[0025] FIG. 5 is a diagram illustrating an example of a process for performing object recognition according to one embodiment of the present disclosure.

[0026] FIGS. 6a and FIGS. 6b are screens of a pill identification application according to one embodiment of the present disclosure.

[0027] FIG. 7 is a flowchart of the operation of an electronic device according to one embodiment of the present disclosure.

[0028] FIG. 8 is a flowchart of the operation of an electronic device according to one embodiment of the present disclosure.

[0029] FIGS. 9 and FIGS. 10 are drawings illustrating an image in which at least one graphic object is overlaid on at least one pill according to one embodiment of the present disclosure.

[0030] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0031] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0032] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0033] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0034] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0035] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0036] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some contexts, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0037] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.

[0038] Additionally, in the following embodiments, where it is stated that one component is 'connected', 'coupled', or 'joined' to another component, it should be understood that the component may be directly connected or joined to the other component, but that another component may also be 'connected', 'coupled', or 'joined' between each component.

[0039] In the present disclosure, 'each of the plurality of A' may refer to each of all components included in the plurality of A, or each of some components included in the plurality of A.

[0040] Additionally, the terms 'comprises' and / or 'comprising' as used in the following embodiments do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.

[0041] In the present disclosure, "pill" may refer to a solid formulation having a specific shape. For example, the "pill" may be made of compressed powder or granules and may maintain a formulation form containing one or more active ingredients. Additionally, the pill may be provided in various sizes and shapes. For example, the pill may be in various shapes such as round, oval, square, or cylindrical, or may be in a shape broken from a solid formulation. Additionally, the pill may be in various colors depending on its material and ingredients. Additionally, there may be an imprint, such as a printed material, on one or both sides of the pill. The imprint may include information such as a logo, drug name, dosage, or manufacturer. Additionally, the surface of the pill may have other surface marks, such as scratches or cracks. For example, the pill may be implemented in formulations such as tablets, extended-release tablets, gastro-resistant tablets, triple tablets, orally disintegrating tablets, sugar-coated tablets, troches, vaginal suppositories, enteric-coated pills, capsules, soft capsules, etc., but is not limited thereto. The pill may be packaged in blisters, pouches, sachets, bottles, etc.

[0042] In the present disclosure, the 'recognition result' is information generated by an object recognition model and may include location information, size information, width information, height information, bounding box information, etc. of the recognized object. Additionally, the recognition result may include information generated by the object recognition model based on an image and or an image and prescription data. Additionally, the recognition result may include confidence values ​​and / or category information.

[0043] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0044] FIG. 1 is a schematic diagram illustrating an example of an electronic device according to one embodiment of the present disclosure. As illustrated, the electronic device (100) may include a processor (110), a camera (120), and a display (130).

[0045] A processor (110) according to one embodiment can, for example, execute software to control at least one other component (e.g., hardware or software component) of an electronic device (100) connected to the processor (110) and can perform various data processing or operations.

[0046] A camera (120) according to one embodiment can capture still images and video. According to one embodiment, the camera (120) may include one or more lenses, image sensors, image signal processors, or flashes.

[0047] The electronic device (100) can identify at least one pill included in an image captured through a camera (120). Although FIG. 1 is illustrated as identifying pills using one camera (120), it is not limited thereto, and two or more cameras may be used.

[0048] According to one embodiment, the camera (120) can generate an image containing at least one pill. For example, the camera (120) can generate an image containing at least one pill placed on a tray. An image containing at least one pill captured by the camera (120) can be transmitted to a processor (110). The image generated by the camera may be a single still image or a video that is a set of multiple images in a time series.

[0049] A display (130) according to one embodiment can visually provide information to an external (e.g., user) of an electronic device (100). The display (130) may include a touch circuit configured to detect a touch, or a sensor circuit configured to measure the intensity of a force generated by a touch.

[0050] The user can place a tray containing pills to be identified onto the FoV of the camera (120) of the electronic device (100). For example, the user can place the tray containing pills onto the focal length of the camera (120). The user can take an image containing at least one pill placed on the tray.

[0051] According to one embodiment, the processor (110) can generate a recognition result by performing object recognition on an image using an object recognition model. The processor (110) can identify at least one pill included in the image using an object recognition model.

[0052] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) and a plurality of user terminals (210_1, 210_2, 210_3) are connected to enable communication between them according to one embodiment of the present disclosure. As illustrated, the plurality of user terminals (210_1, 210_2, 210_3) may be connected to an information processing system (230) capable of providing a pill identification service through a network (220). Here, the plurality of user terminals (210_1, 210_2, 210_3) may include a user terminal receiving the pill identification service.

[0053] According to one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data associated with providing pill identification services, etc., or one or more distributed computing devices and / or distributed databases based on cloud computing services.

[0054] The pill identification service provided by the information processing system (230) may be provided to the user through a pill identification application installed on each of the multiple user terminals (210_1, 210_2, 210_3). For example, the information processing system (230) may provide information related to pill identification received from the user terminals (210_1, 210_2, 210_3) through the pill identification application, etc., or perform corresponding processing.

[0055] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) through a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication devices and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication network, wired internet, wireless internet, broadcasting network, satellite network, etc.) but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0056] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto. The user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and running instant messaging applications or web browsers. For example, user terminals may include an AI speaker, a smartphone, a mobile phone, a navigation system, a computer, a laptop, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, a game console, a wearable device, an IoT (Internet of Things) device, a VR (Virtual Reality) device, an AR (Augmented Reality) device, a set-top box, etc. Additionally, FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with an information processing system (230) through a network (220), but is not limited thereto, and may be configured so that a different number of user terminals communicate with an information processing system (230) through a network (220).

[0057] According to various embodiments, the electronic device (100) that performs the method of identifying a pill may be a user terminal (210_1, 210_2, 210_3).

[0058] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to an embodiment of the present disclosure. The user terminal (210) may refer to any computing device capable of executing a pill identification application, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) of FIG. 2. As illustrated, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data through a network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) through the input / output interface (318).

[0059] The memory (312, 332) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as ROM (read-only memory), a disk drive, an SSD (solid-state drive), or a flash memory. As another example, a permanent mass storage device such as ROM, an SSD, a flash memory, or a disk drive may be included in the user terminal (210) or the information processing system (230) as a separate permanent storage device distinct from the memory. Additionally, the memory (312, 332) may store an operating system and at least one program code (e.g., code for a pill identification application installed and running on the user terminal (210)).

[0060] These software components may be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include a recording medium that can be directly connected to the user terminal (210) and the information processing system (230), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, the software components may be loaded into memory (312, 332) via a communication module rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program installed by files provided through a network (220) by developers or a file distribution system that distributes installation files for the application.

[0061] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0062] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system). For example, a request or data (e.g., an image containing at least one pill) generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) can be received by the user terminal (210) through the communication module (336) and the network (220) via the communication module (316) of the user terminal (210).

[0063] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may include a device such as a display (e.g., the display (130) of FIG. 1), a speaker, or a haptic feedback device. As another example, the input / output interface (318) may be a means for interfacing with a device in which the configuration or function for performing input and output is integrated into one, such as a touchscreen. For example, when the processor (314) of the user terminal (210) processes instructions of a computer program loaded in memory (312), a service screen configured using information and / or data provided by an information processing system (230) or another user terminal may be displayed on a display through the input / output interface (318). In FIG. 3, the input / output device (320) is depicted as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). Additionally, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that may be connected to the information processing system (230) or included in the information processing system (230). In FIG. 3, the input / output interface (318, 338) is depicted as an element configured separately from the processor (314, 334), but is not limited thereto and may be configured so that the input / output interface (318, 338) is included in the processor (314, 334).

[0064] The user terminal (210) and the information processing system (230) may include more components than those of FIG. 3. However, it is not necessary to clearly illustrate most of the prior art components. According to one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. Additionally, the user terminal (210) may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and may be implemented to further include various components such as an accelerometer, a gyroscope, an image sensor, a proximity sensor, a touch sensor, an ambient light sensor, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0065] While a program for a pill identification application, etc. is running, the processor (314) can receive text, images, video, voice and / or actions, etc. that are input or selected through an input device such as a touch screen, keyboard, audio sensor and / or image sensor, camera, microphone, etc. connected to an input / output interface (318), and can store the received text, images, video, voice and / or actions, etc. in memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220).

[0066] The processor (314) of the user terminal (210) may be configured to manage, process, and / or store information and / or data received from an input / output device (320), another user terminal, an information processing system (230), and / or a plurality of external systems. The information and / or data processed by the processor (314) may be provided to the information processing system (230) through a communication module (316) and a network (220). The processor (314) of the user terminal (210) may transmit information and / or data to the input / output device (320) through an input / output interface (318) to output it. For example, the processor (314) may display the received information and / or data on the screen of the user terminal (210).

[0067] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals (210) and / or a plurality of external systems. The information and / or data processed by the processor (334) may be provided to the user terminals (210) through a communication module (336) and a network (220).

[0068] FIG. 4 is an operation flowchart (400) of an electronic device according to one embodiment of the present disclosure.

[0069] Referring to the operation flowchart (400), the processor (110) of the electronic device (100) according to one embodiment may receive an image containing at least one pill through a camera (120) in step 410. In one example, a user may place at least one pill on a tray and photograph at least one pill through the camera (120) of the electronic device (100). The camera (120) may generate an image containing at least one pill. The image containing at least one pill may be a still image, a video, or a part of a video. The camera (120) may transmit the generated image containing at least one pill to the processor (110).

[0070] A processor (110) according to one embodiment can display an image through a display (130) in step 420. In one example, the processor (110) can control the display (130) to display an image including at least one pill.

[0071] A processor (110) according to one embodiment can identify at least one pill on an image in step 430. The processor (110) can identify at least one pill on the image using an object recognition model. The processor (110) can identify the number and location of at least one pill. The processor (110) can identify at least one of the size and color of at least one pill. The object recognition model will be described in detail using FIG. 5.

[0072] FIG. 5 is a diagram illustrating an example of a process for performing object recognition according to one embodiment of the present disclosure. The image (510) may be an image of a pill containing at least one pill. For example, the image (510) may be one of images captured through a camera (120) of an electronic device (100).

[0073] The object recognition model (520) can recognize at least one pill included in the image (510) as an object and generate a recognition result (530). For example, the object recognition model (520) may be a neural network model, RetinaNet, YOLO, SSD, R-CNN, etc. The recognition result (530) may include information regarding each object recognized by the object recognition model (520). For example, the recognition result (530) may include location information, size information (width information, height information), category information, confidence value, etc. regarding the bounding box of a specific object.

[0074] According to one embodiment, the object recognition model (520) can learn various pharmaceutical data. Specifically, the object recognition model (520) can learn pharmaceutical data by classifying it by type of pill. For example, the pharmaceutical data may include the name, front image, back image, size information, etc. of each pill. The object recognition model (520) can recognize pill objects within the image (510) based on the learned data. In this case, the category information included in the recognition result (530) may correspond to specific pharmaceutical data (e.g., Tylenol Tablet 500mg (Acetaminophen)).

[0075] According to another embodiment, the object recognition model (520) cannot specify what kind of medicine the pill object in the image (510) is, but can distinguish and recognize different pill shapes included in the image (510). For example, if the image (510) includes a circular pill, a triangular pill, and an elliptical pill, the category information included in the recognition result (530) may correspond to one of the different pill shapes.

[0076] Returning to FIG. 4, a processor (110) according to one embodiment may, in step 440, overlay at least one graphic object on at least one pill. The graphic object may be part of a graphical user interface (GUI) that is displayed to make the user recognize the pill on the image. The size, color, and shape of the graphic object may be determined based on the identified pill and / or may be changed by user input. There may be a one-to-one correspondence between at least one graphic object and at least one pill. For example, if there are 10 pills on the image, 10 graphic objects may be created. The processor (110) may control that at least one graphic object is superimposed and displayed on at least one identified pill on the image.

[0077] FIGS. 6a and 6b are a screen of a pill identification application according to one embodiment of the present disclosure. Referring to FIGS. 6a and 6b, a processor (110) of an electronic device (100) may display a screen of an application for pill identification through a display (130). The screen (600) may include an image selection area (610), a pill information display area (620), a pill image display area (630), and a graphic object attribute selection area (640).

[0078] According to one embodiment, the image selection area (610) may be an area for selecting the type of image for pill identification. For example, if the user wants to identify a pill included in a still image (or image), the user may select the snap icon (611), and if the user wants to identify a pill included in a video containing a series of images, the user may select the Live icon (613). Based on the type selected in the image selection area (610), the camera (120) may capture a still image or a video as an image containing at least one pill.

[0079] According to one embodiment, the pill information display area (620) may be an area that displays information about a pill identified through object recognition. For example, the information about the identified pill may include the type of pill, expiration date, manufacturing information, etc.

[0080] According to one embodiment, the pill image display area (630) may be an area for displaying an image containing at least one pill (P) received from the camera (120). If a video is selected by user input, the processor (110) may display a real-time video of an image containing at least one pill (P) captured through the camera (120) in the pill image display area (630). If a still image is selected by user input, the processor (110) may display a still image containing at least one pill (P) captured through the camera (120) in the image display area (630).

[0081] Referring to FIG. 6b, a processor (110) according to one embodiment may display at least one graphic object (GO) overlaid on at least one pill (P) through a display (130). At least one graphic object (GO) may correspond one-to-one with at least one pill (P). As shown in FIG. 6b, if 11 pills exist (or are identified) on an image, 11 graphic objects may be displayed. The shape of at least one graphic object may be one of a circle, an ellipse, and a polygon. For convenience of explanation, the graphic object is described as being circular (i.e., dot shape) in this disclosure, but the shape of the graphic object is not limited thereto.

[0082] A graphic object attribute selection area (640) according to one embodiment may be an area for changing the attributes of a graphic object. A user can change the color and size of a graphic object through the graphic object attribute selection area (640). A user can change the color and size of a graphic object set to default values.

[0083] According to one embodiment, the size of at least one graphic object may be determined based on the size of at least one pill. The size of at least one graphic object determined based on the size of at least one pill may be set as a default value. The size of at least one graphic object may be determined based on the width information and height information of at least one pill. For example, the processor (110) may check the width information and height information of all or part of at least one pill. For each of all or part of at least one pill, the processor (110) may check the minimum value among the width information and height information and calculate a final value by multiplying the checked minimum value by 0.5. Based on the final values ​​calculated for each of all or part of at least one pill, the processor (110) may calculate the average value thereof. The processor (110) may determine the calculated average value as the diameter of the graphic object. As another example, the processor (110) can calculate an average value of area values ​​by multiplying width information and height information for each of at least one pill or part thereof, and determine the value obtained by multiplying the calculated average value by a predetermined coefficient as the diameter of the graphic object.

[0084] According to one embodiment, the color of at least one graphic object may be determined based on the color of at least one pill. The color of at least one graphic object determined based on the color of at least one pill may be set as a default value. A specific method for determining the color of a graphic object will be explained using FIGS. 7 and 8.

[0085] According to one embodiment, at least one graphic object can be placed in the center of the pill.

[0086] FIG. 7 is an operation flowchart (700) of an electronic device according to one embodiment of the present disclosure. Specifically, FIG. 7 relates to a method for selecting the color of a graphic object.

[0087] Referring to the operation flowchart (700), the processor (110) of the electronic device (100) according to one embodiment can check the color value of the center point of at least one pill in step 710. The color value may be, for example, an RGB value.

[0088] A processor (110) according to one embodiment may determine a representative color value based on the color value of the center point of at least one pill in step 720. The processor (110) may determine the average value or the mode value of the color values ​​of the center points of each of at least one pill as the representative color value.

[0089] A processor (110) according to one embodiment may determine the complementary color value of a representative color value in step 730. The complementary color value may be the RGB value of a color corresponding to the complementary color of the color having the representative color value.

[0090] A processor (110) according to one embodiment can determine, in step 740, a color with maximized saturation in a complementary color value as the color of at least one graphic object.

[0091] FIG. 8 is an operation flowchart (800) of an electronic device according to one embodiment of the present disclosure. Specifically, FIG. 8 relates to a method for selecting the color of a graphic object.

[0092] Referring to the operation flowchart (800), the processor (110) of the electronic device (100) according to one embodiment can check the color value of the center point of at least one pill in step 810. The color value may be, for example, an RGB value.

[0093] A processor (110) according to one embodiment may determine a representative color value based on the color value of the center point of at least one pill in step 820. The processor (110) may determine the average value or the mode value of the color values ​​of the center points of each of at least one pill as the representative color value.

[0094] A processor (110) according to one embodiment may determine the complementary color value of a representative color value in step 830. The complementary color value may be the RGB value of a color corresponding to the complementary color of the color having the representative color value.

[0095] A processor (110) according to one embodiment may, in step 840, determine one color among a plurality of predetermined colors that is most similar to a color value with a complementary color value as the color of at least one graphic object. For example, the processor (110) may determine a color having a color value closest to a complementary color value among a plurality of predetermined colors (e.g., green, orange, yellow) as the color of at least one graphic object.

[0096] FIG. 9 illustrates an image in which at least one graphic object (GO) is overlaid on at least one pill (P) according to one embodiment of the present disclosure. The size of the graphic object (GO) may be set to be smaller than the size of the pill (P) as a default value. The graphic object (GO) may be displayed superimposed on the center point of the pill (P). The color of the graphic object (GO) may be a color with maximized saturation at the complementary color value of the color value of the pill (P). For example, if the color of the pill (P) is green, the color of at least one graphic object (GO) may be set to orange as a default value.

[0097] FIG. 10 is a drawing illustrating an image in which at least one graphic object (GO) is overlaid on at least one pill (P) according to one embodiment of the present disclosure. The size and color of the graphic object (GO) can be changed by user input. For example, the user can change the size or color of the graphic object (GO) through a graphic object (GO) attribute selection area (e.g., area (640) of FIG. 6).

[0098] The above-described flowchart and description are merely examples and may be implemented differently in some embodiments. For instance, in some embodiments, the order of each step may be changed, some steps may be repeated, some steps may be omitted, or some steps may be added.

[0099] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a computer-executable program, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0100] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0101] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0102] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0103] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and may cause the processor(s) to perform specific aspects of the functions described in this disclosure.

[0104] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available media accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0105] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disk and disc include CD, laser disc, optical disc, DVD (digital versatile disc), floppy disk, and Blu-ray disc, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0106] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0107] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0108] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

Claims

1. A method for identifying pills performed by at least one processor, A step of receiving an image containing at least one pill through a camera; A step of displaying the above image through a display; A step of identifying at least one pill on the image above; and Step of overlaying at least one graphic object on at least one pill A method for identifying pills, including 2. In Paragraph 1, The step of identifying at least one pill above is, A step of identifying the number and location of at least one pill; and Step of identifying at least one of the size and color of at least one of the above-mentioned pills A method for identifying pills, including 3. In Paragraph 2, A method for identifying pills, wherein the size of at least one graphic object is determined based on the size of at least one pill.

4. In Paragraph 3, A method for identifying pills, wherein the size of at least one graphic object is determined based on width information and height information of at least one pill.

5. In Paragraph 2, A method for identifying pills, wherein the color of at least one graphic object is determined based on the color of at least one pill.

6. In Paragraph 5, A step of checking the color value of the center point of at least one pill; A step of determining a representative color value based on the color value of the center point of at least one pill; Step of determining the complementary color value of the above representative color value; and Step of determining the color with maximized saturation of the above complementary color value as the color of the above at least one graphic object A method for identifying pills that further include 7. In Paragraph 5, A step of checking the color value of the center point of at least one pill; A step of determining a representative color value based on the color value of the center point of at least one pill; Step of determining the complementary color value of the above representative color value; and Step of determining one color among a plurality of predetermined colors that is most similar to the color value of the complementary color as the color of the at least one graphic object A method for identifying pills that further include 8. In Paragraph 2, A method for identifying pills in which the above at least one graphic object and the above at least one pill correspond one-to-one.

9. In Paragraph 1, A step of changing the size of at least one graphic object based on a first user input; and A step of changing the color of at least one graphic object based on a second user input A method for identifying pills that further include 10. In Paragraph 1, A method for identifying a pill, wherein the shape of at least one graphic object is one of a circle, an ellipse, and a polygon.

11. A computer program stored on a computer-readable recording medium for executing a method according to any one of paragraphs 1 through 10 on a computer.

12. In electronic devices, camera; Display; and Includes a processor, The above processor is, An image containing at least one pill is received through the above camera, and Display the image through the above display, and Identifying at least one pill on the above image, and An electronic device configured to overlay at least one graphic object on at least one pill.

Images