Method and apparatus for recognizing products using digital image

Abstract

Disclosed are a method and an apparatus for recognizing products using a digital image. The method for recognizing products using a digital image, according to one embodiment of the present invention, may comprise the steps of: receiving photographs taken for a plurality of products from a product recognition terminal; receiving weight information obtained by summing weights corresponding to the plurality of products using a weight sensor; recognizing the plurality of products on the basis of the photographs and the weight information; and providing a recognition result for the plurality of products to a user.

Description

Product recognition method and device using digital images

[0001] The present invention relates to a method and device for recognizing a product using a digital image.

[0002] The content described below merely provides background information related to one embodiment of the present invention and does not constitute prior art.

[0003] Existing product management in stores and warehouses suffers from high product waste rates, lost sales opportunities due to out-of-stock items, and increased inventory management costs due to excess inventory. Existing inventory management methods require individual counts and manual entry of each item, which requires significant time and human resources. Barcode scanners are used to improve this, but even these methods require tagging every item, limiting their effectiveness. Therefore, a product recognition method that can simultaneously recognize multiple items and reduce scanning time is needed.

[0004] The purpose of the present invention is to recognize a product using a digital image.

[0005] In addition, the present invention aims to simultaneously photograph and recognize multiple products.

[0006] A product recognition method using a digital image according to one embodiment of the present invention for achieving the above-described purpose may include a step of receiving photos taken of a plurality of products from a product recognition terminal, a step of receiving weight information obtained by adding up the weights corresponding to the plurality of products using a weight sensor, a step of recognizing the plurality of products based on the photos and the weight information, and a step of providing a user with recognition results for the plurality of products.

[0007] The above plurality of products may include a digital image corresponding to information about each of the plurality of products, and the photo may include first identification information corresponding to the overall shape of each of the plurality of products and second identification information corresponding to the digital image.

[0008] The step of recognizing the plurality of products may include a step of generating a first product candidate group according to the shape of each of the plurality of products based on the first identification information, a step of generating a second product candidate group according to a digital image of each of the plurality of products based on the second identification information, a step of generating a product combination from which a weight corresponding to the weight information can be calculated by applying a preset weight based on the first product candidate group and the second product candidate group, and a step of generating a product recognition result including the product combination.

[0009] Before the step of receiving the above photo, a step of providing a grid-shaped guide device that guides the user to place the plurality of products in a well-distributed manner may be included.

[0010] After the step of recognizing the plurality of products, if there is an unrecognized product among the plurality of products, the step of calculating a first zone in which the unrecognized product exists among the zones divided in a grid shape in the guide device and the step of controlling the first light installed to shine toward the first zone so as to illuminate the unrecognized product may be included.

[0011] In addition, a product recognition device using a digital image according to one embodiment of the present invention may include a product recognition terminal that takes pictures of a plurality of products, a picture receiving unit that receives the pictures from the product recognition terminal, a weight sensor that measures weights corresponding to the plurality of products, a weight information receiving unit that receives weight information measured from the weight sensor, a product recognition unit that recognizes the plurality of products based on the pictures and the weight information, and a recognition result providing unit that provides recognition results for the plurality of products to a user.

[0012] The above plurality of products may include a digital image corresponding to information about each of the plurality of products, and the photo may include first identification information corresponding to the overall shape of each of the plurality of products and second identification information corresponding to the digital image.

[0013] The product recognition unit may, based on the first identification information, calculate a first product candidate group according to the shape of each of the plurality of products, and, based on the second identification information, calculate a second product candidate group according to the digital image of each of the plurality of products, and, based on the first product candidate group and the second product candidate group, calculate a product combination from which a weight corresponding to the weight information can be calculated by applying a preset weight, and generate a product recognition result including the product combination.

[0014] It may further include a grid-shaped guide device that guides the user to place the above multiple products in a well-distributed manner.

[0015] If there is an unrecognized product among the plurality of products, the control unit may further include a control unit that calculates the location of the unrecognized product and provides the location to the user, and if there is an unrecognized product among the plurality of products, the control unit may calculate a first zone in which the unrecognized product exists among zones divided in a grid shape in the guide device, and control the first light installed to shine toward the first zone so that the first light illuminates the unrecognized product.

[0016] According to the present invention, a product can be recognized using a digital image.

[0017] In addition, according to the present invention, it is possible to simultaneously recognize multiple products to increase the product recognition speed.

[0018] FIG. 1 is a block diagram showing entities for product recognition using digital images according to one embodiment of the present invention.

[0019] FIG. 2 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0020] FIG. 3 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0021] Figure 4 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0022] FIG. 5 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0023] FIG. 6 is a drawing showing the configuration of a product recognition device using a digital image according to one embodiment of the present invention.

[0024] FIG. 7 is a diagram illustrating a computer system according to one embodiment of the present invention.

[0025] The present invention will be described in detail with reference to the attached drawings. Herein, repetitive descriptions, well-known functions that may unnecessarily obscure the gist of the present invention, and detailed descriptions of configurations are omitted. The embodiments of the present invention are provided to more fully explain the present invention to those of ordinary skill in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clarity.

[0026] Although "first" or "second" are used to describe various components, these components are not limited by such terms. Such terms may only be used to distinguish one component from another. Accordingly, a first component referred to below may also be a second component within the technical scope of the present invention.

[0027] Throughout the specification, whenever a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0028] Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings.

[0029] FIG. 1 is a block diagram showing entities for product recognition using digital images according to one embodiment of the present invention.

[0030] Referring to FIG. 1, in one embodiment of the present invention, the entities for product recognition using digital images include a product recognition device (110) and a product recognition terminal (120).

[0031] The product recognition device (110) may refer to a device that receives photos taken of multiple products from a product recognition terminal (120) and recognizes multiple products through weight information and photos of the multiple products.

[0032] The product recognition device (110) may be a device that performs the function of recognizing products in a store that sells products or a logistics center that handles a large quantity of products.

[0033] The product recognition device (110) may be a device that provides recognition results for multiple products to a user.

[0034] The product recognition terminal (120) may be a device that takes pictures of multiple products to be recognized and transmits the pictures to the product recognition device (110).

[0035] A product recognition terminal (120) refers to a communication terminal capable of utilizing a terminal application in a wired or wireless communication environment. For example, the product recognition terminal (120) may be a portable terminal such as a smart phone or a personal computer. However, the spirit of the present invention is not limited thereto, and any terminal equipped with a terminal application may be utilized without limitation.

[0036] Here, the terminal application may be a dedicated application for accessing the product recognition device (110) or a web browser that can access the product recognition device (110) on a web page.

[0037] The product recognition device (110) and the product recognition terminal (120) can be interconnected through a communication network.

[0038] A communications network refers to a connection path that enables data to be transmitted and received between the above entities. For example, a communications network may encompass wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), or wireless networks such as wireless LANs, CDMA, Bluetooth, and satellite communications. However, the scope of communications networks applicable to the present invention is not limited thereto.

[0039] FIG. 2 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0040] Referring to FIG. 2, a product recognition method using a digital image according to one embodiment of the present invention can first receive photos taken of multiple products from a product recognition terminal (S210).

[0041] Here, the plurality of products may include digital images corresponding to information about each of the plurality of products.

[0042] Additionally, the photograph may include first identification information corresponding to the overall shape of each of the plurality of products and second identification information corresponding to the digital image.

[0043] Next, weight information obtained by adding up the weights corresponding to the plurality of products can be received using a weight sensor (S220).

[0044] Next, based on the above photo and the above weight information, the plurality of products can be recognized (S230).

[0045] Next, the recognition results for the above multiple products can be provided to the user (S240).

[0046] FIG. 3 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0047] Referring to FIG. 3, a product recognition method using a digital image according to one embodiment of the present invention can first, based on the first identification information, produce a first product candidate group according to the shape of each of the plurality of products (S310).

[0048] Next, based on the second identification information, a second product candidate group can be derived according to the digital image of each of the plurality of products (S320).

[0049] Next, a product combination for which a weight corresponding to the weight information can be calculated can be calculated by applying preset weights based on the first product candidate group and the second product candidate group (S330).

[0050] Next, a product recognition result including the above product combination can be generated (S340).

[0051] For example, a product recognition device can generate a first set of product candidates based on the shape of each of multiple products and a second set of product candidates based on digital images. Furthermore, the device can combine products included in the first set of product candidates with products included in the second set of product candidates to generate a product combination that can produce a weight corresponding to the weight information, and generate a product recognition result including this combination.

[0052] Figure 4 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0053] Referring to FIG. 4, a product recognition method using a digital image according to one embodiment of the present invention can provide a grid-shaped guide device that guides a user to place the plurality of products in a well-distributed manner before the step of receiving the photo (S410).

[0054] Here, the guide device can be arranged in a grid shape with a size equal to or smaller than the product recognition terminal's ability to capture product images. The guide device allows the user to place multiple products evenly distributed across the grid-shaped areas, effectively increasing the product recognition rate for multiple products.

[0055] Additionally, the guide device can be arranged in a grid shape on a plate such as glass or acrylic that has a certain level of transparency or higher to allow light to pass through.

[0056] FIG. 5 is a flowchart illustrating a product recognition method using a digital image according to one embodiment of the present invention.

[0057] Referring to FIG. 5, a product recognition method using a digital image according to one embodiment of the present invention can, after the step of recognizing the plurality of products, if there is an unrecognized product among the plurality of products, calculate a first zone in which the unrecognized product exists among zones divided in a grid shape in the guide device (S510).

[0058] Next, the first light installed to shine toward the first zone can be controlled to illuminate the unrecognized product (S520).

[0059] For example, if an unrecognized product is found among multiple products, the first zone where the product is located can be calculated among the zones divided into a grid. Next, the first light, installed to illuminate the calculated first zone, can be controlled to operate.

[0060] At this time, the first light may be installed so as to shine from the upper part of the guide device toward the first area of ​​the guide device, or may be installed so as to shine from the lower part of the guide device toward the first area of ​​the guide device so as to be displayed to the user.

[0061] In one embodiment, if one of the multiple products is not recognized because it is placed on a line dividing the grid pattern on the guide device, the remaining products may be recognized but may not match the weight information. In such cases, a notification may be provided to the user to confirm that the products are placed on the line dividing the grid pattern, allowing the user to redistribute and place the multiple products on the guide device.

[0062] As an optional embodiment, when taking pictures of multiple products using a product recognition terminal, a photo score can be calculated and provided to the user.

[0063] Specifically, while a dedicated application for acquiring photos is running on a product recognition terminal, images can be acquired in real time and the shapes of each of multiple products can be identified on the images.

[0064] Next, when the shape of each of the multiple products is identified, a photo score can be calculated from the image acquired in real time.

[0065] Here, the photo score can be expressed as the sum of the first partial score and the second partial score.

[0066] Additionally, the first partial score represents the recognizability of the digital images included in each of the multiple products in the images acquired in real time, and the clearer the digital images included in each of the multiple products, the higher the first partial score can be.

[0067] Additionally, the second part score may decrease as the degree of slant of each of the multiple products becomes more severe.

[0068]

[0069] *For example, if a digital image is clearly recognized as close to the front as possible in an image received from a product recognition terminal, the possibility of recognizing the digital image increases, and thus the first part score may increase.

[0070] Additionally, if the image is taken at an angle, the recognizability of the digital image may be reduced, which may result in a decrease in the second part score.

[0071] Next, the product recognition terminal can be provided with the possibility of product recognition corresponding to the photo score.

[0072] Next, if the photo score exceeds a preset standard score, the product recognition accuracy can be provided in real time to the product recognition terminal.

[0073] For example, a user can take a photo for product recognition while checking the product recognition possibility and product recognition accuracy in real time through a product recognition terminal.

[0074] As an optional embodiment, the first partial score may be calculated for each image acquired at a preset cycle and accumulated to the current photo score, but the total accumulated score of the first partial score may not exceed a preset first partial score maximum.

[0075] In this case, the photo score may initially start with a score below a reference score, and as images are acquired in real time, a first partial score may be added at a preset interval to exceed the reference score, and the sum of the initial score and the maximum first partial score may be set higher than the reference score.

[0076] Additionally, the second partial score may be calculated for each image acquired at a preset cycle and accumulated to the current photo score, but the total accumulated score of the second partial score may not exceed the preset second partial score maximum.

[0077] In this case, the photo score may initially start with a score below a reference score, and as images are acquired in real time, a second partial score may be added at a preset interval to exceed the reference score, and the sum of the initial score and the maximum second partial score may be set higher than the reference score.

[0078] Through this, the present invention can provide a more natural user experience by visually providing the user with a process in which the photo score gradually increases to reach a reference score during the process of acquiring a number of photos for accurate measurement information.

[0079] As an optional embodiment, the color of the first light can be controlled to vary depending on the color of the unrecognized product.

[0080] Specifically, after calculating the first zone where the unrecognized product exists, first color information about the unrecognized product can be obtained.

[0081] Here, the first color information may include the first color that occupies the highest proportion in the color spectrum of the unrecognized product.

[0082] Next, we can derive a second color that is complementary to the first color on the color wheel.

[0083] Next, the first light can be controlled to illuminate the unrecognized product in the second color.

[0084] For example, if the primary color with the highest percentage of unrecognized products is red, the product recognition device can recognize the primary color red and derive cyan, the complementary color of red on the color wheel. Next, the primary light can be controlled to illuminate the unrecognized products in cyan.

[0085] As an optional embodiment, if there are multiple unrecognized products, the first light can be controlled to illuminate each of the multiple unrecognized products with a complementary color of the color of each product.

[0086] Specifically, second color information and third color information can be obtained for each of the first product and the second product, which are multiple unrecognized products.

[0087] Here, the second color information may include a third color having the highest proportion in the color spectrum of the first product, and the third color information may include a fourth color having the highest proportion in the color spectrum of the second product.

[0088] Next, we can derive a fifth color that is complementary to the third color on the color wheel, and a sixth color that is complementary to the fourth color.

[0089] Next, the first light can be controlled to split the first light so that one side of the first light illuminates the first product with the fifth color, and the other side of the first light illuminates the second product with the sixth color.

[0090] At this time, the first light may include a form in which a plurality of small lights are combined.

[0091] For example, if the third color with the highest proportion in the first product is red and the fourth color with the highest proportion in the second product is yellow, the product recognition device can recognize the third color red, output cyan, which is the complementary color of red on the color wheel, recognize the fourth color yellow, and output indigo, which is the complementary color of yellow on the color wheel. Next, the first light can be controlled so that one side of the first light illuminates the first product with cyan, and the other side illuminates the second product with indigo.

[0092] As an optional embodiment, when a plurality of unrecognized products are adjacent to one of the zones divided into a grid shape in the guide device, the color of the lighting can be controlled according to the difference in wavelength corresponding to the color information of each product.

[0093] Specifically, when multiple unrecognized products, the third product and the fourth product, are adjacent to each other in one of the areas divided in a grid shape, the fourth color information and the fifth color information for each of the third products can be obtained.

[0094] Here, the fourth color information may include the seventh color having the highest proportion in the color spectrum of the third product, and the fifth color information may include the eighth color having the highest proportion in the color spectrum of the fourth product.

[0095] Next, the difference in wavelength for each of the seventh and eighth colors can be calculated.

[0096] Next, if the difference in wavelength is within a preset standard, that is, if the seventh color and the eighth color are adjacent within a preset standard on the color wheel, a ninth color that is complementary to the color that occupies a larger proportion of the seventh color and the eighth color can be produced.

[0097] Next, the first light can be controlled to illuminate multiple unrecognized products together in a ninth color.

[0098] Conversely, if the difference in wavelength exceeds a preset standard, i.e., if the seventh and eighth colors are further apart than a preset standard on the color wheel, a ninth and tenth color, which are complementary colors of the seventh and eighth colors, respectively, can be produced.

[0099] At this time, the first light can be controlled to illuminate multiple unrecognized products together by changing the first light to the ninth color and the tenth color at regular intervals.

[0100] For example, if the seventh and eighth colors are red and orange, and the color that occupies a high proportion of the two colors is red, since red and orange are colors that are closely adjacent to each other on the color wheel, the product recognition device can output cyan, which is the complementary color of red, and control the first light so that the first light illuminates a plurality of unrecognized products together in cyan.

[0101] Additionally, if the seventh and eighth colors are red and blue, since red and blue are distant colors on the color wheel, the product recognition device can output cyan, the complementary color of red, and orange, the complementary color of blue. Furthermore, the first light can be controlled to illuminate multiple unrecognized products simultaneously by periodically switching between cyan and orange.

[0102] This has the effect of allowing users to easily recognize unrecognized products by illuminating them with a color that contrasts with the color of the product.

[0103] FIG. 6 is a drawing showing the configuration of a product recognition device using a digital image according to one embodiment of the present invention.

[0104] Referring to FIG. 6, a product recognition device using a digital image according to one embodiment of the present invention may include a product recognition terminal that takes pictures of a plurality of products, a picture receiving unit that receives the pictures from the product recognition terminal, a weight sensor that measures weights corresponding to the plurality of products, a weight information receiving unit that receives weight information measured from the weight sensor, a product recognition unit that recognizes the plurality of products based on the pictures and the weight information, and a recognition result providing unit that provides recognition results for the plurality of products to a user.

[0105] At this time, the plurality of products may include a digital image corresponding to information about each of the plurality of products, and the photo may include first identification information corresponding to the overall shape of each of the plurality of products and second identification information corresponding to the digital image.

[0106] According to one embodiment, the product recognition unit may calculate a first product candidate group according to the shape of each of the plurality of products based on the first identification information, calculate a second product candidate group according to the digital image of each of the plurality of products based on the second identification information, calculate a product combination for which a weight corresponding to the weight information can be calculated by applying a preset weight based on the first product candidate group and the second product candidate group, and generate a product recognition result including the product combination.

[0107] According to one embodiment, a product recognition device using a digital image may further include a grid-shaped guide device that guides a user to place the plurality of products in a well-distributed manner.

[0108] According to one embodiment, a product recognition device using a digital image may further include a control unit that, if there is an unrecognized product among the plurality of products, calculates the location of the unrecognized product and provides the location to the user.

[0109] At this time, if there is an unrecognized product among the plurality of products, the control unit can calculate a first zone in which the unrecognized product exists among the zones divided in a grid shape in the guide device, and control the first light installed to shine toward the first zone so as to illuminate the unrecognized product.

[0110] FIG. 7 is a diagram illustrating a computer system according to one embodiment of the present invention.

[0111] A product recognition device using a digital image according to one embodiment of the present invention can be implemented in a computer system (1000) such as a computer-readable recording medium.

[0112] Referring to FIG. 7, a computer system (1000) may include one or more processors (1010), memory (1030), a user interface input device (1040), a user interface output device (1050), and storage (1060) that communicate with each other via a bus (1020). In addition, the computer system (1000) may further include a network interface (1070) connected to a network (1080). The processor (1010) may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory (1030) or the storage (1060). The memory (1030) and the storage (1060) may be various forms of volatile or non-volatile storage media. For example, the memory may include ROM (1031) or RAM (1032).

[0113] The specific implementations described in the present invention are merely exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components depicted in the drawings are merely exemplary functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0114] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and not only the scope of the patent claims described below but also all scopes equivalent to or equivalently modified from the scope of the patent claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. A step of receiving photos taken of multiple products from a product recognition terminal; A step of receiving weight information obtained by adding up the weights corresponding to the plurality of products using a weight sensor; A step of recognizing the plurality of products based on the above photo and the above weight information; and A step of providing the user with recognition results for the above multiple products. A method for recognizing a product using a digital image, including:

2. In paragraph 1, The above multiple products are, Including a digital image corresponding to information about each of the above multiple products, The above photo is, A product recognition method using a digital image, comprising first identification information corresponding to the overall shape of each of the plurality of products and second identification information corresponding to the digital image.

3. In paragraph 2, The step of recognizing the above multiple products is: A step of generating a first product candidate group according to the shape of each of the plurality of products based on the first identification information; A step of generating a second product candidate group according to a digital image of each of the plurality of products based on the second identification information; A step of calculating a product combination that can calculate a weight corresponding to the weight information by applying a preset weight based on the first product candidate group and the second product candidate group; and A step of generating a product recognition result including the above product combination. A method for recognizing a product using a digital image, including:

4. In paragraph 3, Before receiving the above photo, A step of providing a grid-type guide device that guides the user to place the above multiple products in a well-distributed manner. A method for recognizing a product using a digital image, including:

5. In paragraph 4, After the step of recognizing the above multiple products, If there is an unrecognized product among the above multiple products, a step of calculating a first zone in which the unrecognized product exists among the zones divided in a grid shape in the guide device; and A step of controlling the first light installed to shine toward the first zone to illuminate the unrecognized product. A method for recognizing a product using a digital image, including:

6. Product recognition terminal that takes pictures of multiple products; A photo receiving unit that receives the photo from the product recognition terminal; A weight sensor for measuring the weight corresponding to the above plurality of products; A weight information receiving unit that receives weight information measured from the above weight sensor; A product recognition unit that recognizes the plurality of products based on the above photo and the above weight information; and A recognition result provision unit that provides the user with recognition results for the above multiple products. A product recognition device using a digital image, including:

7. In paragraph 6, The above multiple products are, Including a digital image corresponding to information about each of the above multiple products, The above photo is, A product recognition device using a digital image, comprising first identification information corresponding to the overall shape of each of the plurality of products and second identification information corresponding to the digital image.

8. In paragraph 7, The above product recognition unit, Based on the first identification information, a first product candidate group is generated according to the shape of each of the plurality of products, Based on the second identification information, a second product candidate group is generated according to the digital image of each of the plurality of products, By applying preset weights based on the first product candidate group and the second product candidate group, a product combination capable of producing a weight corresponding to the weight information is calculated, A product recognition device using a digital image, which generates a product recognition result including the above product combination.

9. In paragraph 8, A grid-type guide device that guides the user to place the above multiple products in a well-distributed manner. A product recognition device using a digital image, further comprising:

10. In paragraph 9, If there is an unrecognized product among the above multiple products, a control unit that calculates the location of the unrecognized product and provides the location to the user. Including more, The above control unit, If there is an unrecognized product among the above multiple products, the guide device calculates a first zone in which the unrecognized product exists among the zones divided in a grid shape, A product recognition device using a digital image, wherein the first light installed to shine toward the first zone is operated to control the first light to illuminate the unrecognized product.

Images