Carrier with color-coded information, commodity identification method and settlement method

Abstract

The present invention discloses a carrier with color coding information, a commodity identification method and a settlement method. The commodity settlement method comprises: obtaining image information for settlement, wherein the image information includes one or more carriers with color coding information, wherein the carrier with color coding information is used to accommodate the commodity to be settled, the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance; according to the color coding information on each carrier in the image information, the settlement information corresponding to each carrier is determined respectively for commodity settlement. According to the embodiment of the present invention, the accuracy of self-service settlement can be improved.

Description

Technical Field

[0001] The invention relates to data processing, and in particular to a carrier with color coding information, a commodity identification method and a settlement method. Background Art

[0002] Catering and daily necessities are indispensable in people's lives. At present, the catering industry and supermarkets generally use manual settlement. Since manual settlement is slow, it is easy to cause calculation errors during peak dining and shopping periods when the flow of people is very large. Therefore, the manual settlement method not only delays time but also affects settlement efficiency.

[0003] As people's pace of life increases, they are increasingly seeking to use self-service checkout to achieve efficiency and time saving. For example, there are currently methods such as dish recognition and plate shape recognition for self-service checkout. However, plate recognition is easily affected by the dish, resulting in a low recognition rate for the plate shape and a low accuracy rate for dish recognition, which is prone to errors. Summary of the invention

[0004] The embodiments of the present invention provide a carrier with color coding information, a commodity identification method and a settlement method, which can improve the accuracy of self-service settlement.

[0005] In one embodiment of the present invention, a commodity settlement method is provided, comprising:

[0006] Acquiring image information for settlement, wherein the image information includes one or more carriers with color coding information, wherein the carriers with color coding information are used to accommodate commodities to be settled, the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance;

[0007] According to the color coding information on each carrier in the image information, the settlement information corresponding to each carrier is determined respectively for commodity settlement.

[0008] In one embodiment, the image information is processed by a deep learning-based target detection algorithm to obtain a bounding box of each carrier in the image information, and the color coding information in each bounding box is extracted by a deep learning algorithm.

[0009] In one embodiment, the deep learning based target detection algorithm includes a single shot multi-box detection (SSD) algorithm and / or a Yolo algorithm.

[0010] In one embodiment, at least two colors are extracted from multiple colors. When extracting, all colors are subjected to contrast processing, and at least two colors are extracted using the principle of maximizing color space distance.

[0011] In one embodiment, the at least two colors include colors extracted from vegetables and / or fruits.

[0012] In one embodiment, the color-coding information corresponds one-to-one with the price information.

[0013] In one embodiment, the carrier is a dinner plate, and the color-coded information is represented by an annular color band of two colors arranged along the edge of the dinner plate.

[0014] In another embodiment of the present invention, a commodity settlement system is provided, comprising: a camera, a carrier with color coding information, and a settlement information processing device, wherein:

[0015] The camera is used to obtain image information for settlement, and the image information includes one or more carriers with color coding information;

[0016] A carrier with color coding information, used for accommodating the commodities to be settled, wherein the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance;

[0017] The settlement information processing device is used to determine the settlement information corresponding to each carrier according to the color coding information on each carrier in the image information, which is used for commodity settlement.

[0018] In another embodiment of the present invention, a commodity settlement device is provided, the device comprising:

[0019] an acquisition module, used to acquire image information for settlement, wherein the image information includes one or more carriers with color coding information, wherein the carriers with color coding information are used to accommodate commodities to be settled, the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance;

[0020] The settlement module is used to determine the settlement information corresponding to each carrier according to the color coding information on each carrier in the image information, for commodity settlement.

[0021] In another embodiment of the present invention, a carrier with color coding information is provided, the carrier is used to accommodate goods to be settled, the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance.

[0022] In one embodiment, at least two colors are extracted from multiple colors. When extracting, all colors are subjected to contrast processing, and at least two colors are extracted using the principle of maximizing color space distance.

[0023] In one embodiment, the carrier is a dinner plate, and the color-coded information is represented by an annular color band of two colors arranged along the edge of the dinner plate.

[0024] In another embodiment of the present invention, a commodity identification method is provided, the method comprising:

[0025] Acquire image information for identification, wherein the image information includes one or more commodity placement areas with color coding information, wherein the commodity placement areas with color coding information are used to place commodities, the color coding information is represented by at least two colors, and a distance between any two colors of the at least two colors in a color space is greater than a predetermined distance;

[0026] According to the color coding information of each commodity placement area in the image information, commodity information corresponding to each commodity placement area is determined for commodity identification.

[0027] In one embodiment, the image information is processed by a deep learning-based target detection algorithm to obtain a bounding box of each product placement area in the image information, and the color coding information in each bounding box is extracted by a deep learning algorithm.

[0028] In one embodiment, the deep learning-based target detection algorithm includes an SSD algorithm and / or a Yolo algorithm.

[0029] In one embodiment, at least two colors are extracted from multiple colors. When extracting, all colors are subjected to contrast processing, and at least two colors are extracted using the principle of maximizing color space distance.

[0030] In one embodiment, the color coding information corresponds to the product information one by one.

[0031] In one embodiment, the one or more product placement areas belong to a product rack or a warehouse, and the color coding information is represented by a two-color annular color band arranged along the edge of the product placement area.

[0032] In another embodiment of the present invention, a computing device is provided, comprising: a processor and a memory storing computer program instructions;

[0033] When the processor executes the computer program instructions, the commodity settlement method provided by the embodiment of the present invention is implemented.

[0034] In another embodiment of the present invention, a computer storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the commodity settlement method provided in the embodiment of the present invention is implemented.

[0035] According to an embodiment of the present invention, the color coding information represented by at least two colors on the carrier is first detected, and then the settlement information of the goods to be settled is determined using the color coding information on the carrier, thereby realizing self-service settlement of the goods to be settled. By using more than two colors to represent the settlement information of the goods, more identification information is provided, and the accuracy of self-service settlement is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following briefly introduces the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 is a flowchart of a commodity settlement method according to an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the principle of a commodity settlement method according to an embodiment of the present invention;

[0039] Figure 3 is a structural block diagram of a commodity settlement system according to an embodiment of the present invention;

[0040] Figure 4 It is a structural block diagram of a commodity settlement device according to an embodiment of the present invention;

[0041] Figure 5 A flowchart of a commodity identification method provided in an embodiment of the present invention;

[0042] Figure 6 The figure is a structural block diagram of an exemplary hardware architecture of a commodity settlement device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0044] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0045] In order to better understand the present invention, the implementation of the present invention is described in detail below through specific embodiments. These embodiments are not intended to limit the scope of protection of the present invention.

[0046] Figure 1 FIG. 1 is a flow chart of a commodity settlement method 100 according to an embodiment of the present invention. Figure 1 As shown, the commodity settlement method provided by the embodiment of the present invention includes the following steps:

[0047] S110, obtaining image information for settlement, wherein the image information includes one or more carriers with color coding information, wherein the carriers with color coding information are used to accommodate goods to be settled, the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance.

[0048] S120, determining the settlement information corresponding to each carrier according to the color coding information on each carrier in the image information for commodity settlement.

[0049] The commodity settlement method provided by the embodiment of the present invention determines the settlement information of the commodity to be settled by using the color coding information represented by at least two colors on the carrier. Compared with a single color, it has more identification information and is not easily affected by the color of the commodity to be settled itself during settlement, thereby improving the accuracy of self-service settlement.

[0050] In the embodiment of the present invention, the carrier with color coding information is the settlement product with color coding information. The specific implementation process of the commodity settlement method will be described in detail below using the settlement product instead of the carrier.

[0051] Among them, settlement supplies are used to hold goods to be settled. Taking the scenario of a fast food restaurant as an example, when a user is dining in a fast food restaurant, the food selected by the user is the goods to be settled, and the plates, bowls and other tableware that hold the food selected by the user are settlement supplies.

[0052] In an embodiment of the present invention, the settlement supplies may be any one of the following settlement supplies: a plate, a product packaging box, or a product packaging bag. Taking a supermarket as an example, when a user goes shopping in a supermarket, the product selected by the user is the product to be settled, and the packaging box or packaging bag containing the product to be settled is the settlement supply.

[0053] The color coding information on the settlement supplies can be represented by a settlement mark pre-set on the settlement supplies. That is, by using the settlement mark pre-set on the product packaging box or product packaging bag, the settlement information of the to-be-settled product can be obtained. The embodiment of the present invention does not specifically limit the type and shape of the settlement supplies.

[0054] In some examples, the image information of the settlement supplies can be obtained from an image acquisition device. For example, the image acquisition device can be a camera that captures the image information.

[0055] In some examples, before acquiring the image information of the settlement supplies, the image acquisition device first determines whether there are settlement supplies placed in the designated acquisition area. If it is determined that there are settlement supplies placed, the image acquisition device is started to capture the image information of the settlement supplies in the designated acquisition area.

[0056] In actual application, when there are settlement supplies placed in the designated collection area of ​​the image collection device, the image collection device is activated to capture the image information of the settlement supplies. When there are no settlement supplies placed in the designated collection area, the image collection device may not be activated to save resources and ensure that there are settlement supplies in the image collected by the image collection device.

[0057] Therefore, users who need to settle goods can place the settlement supplies containing the goods to be settled in the designated collection area of ​​the image acquisition device. When the high-resolution camera detects that the settlement supplies are placed in the designated collection area, it will take a picture of the settlement supplies in the designated collection area to obtain high-resolution image information of the settlement supplies.

[0058] After the image information of the settlement supplies is obtained, the settlement information of the goods to be settled contained in the settlement supplies needs to be determined according to the image information of the settlement supplies. In some embodiments of the present invention, step S110 includes the following steps:

[0059] S1101, using a deep learning-based target detection algorithm to process the image information, to obtain a bounding box for each carrier in the image information.

[0060] Among them, the bounding box refers to a geometric body with a slightly larger volume and simple characteristics, which is used to approximate complex geometric objects. The bounding box of an object is a closed space that completely encloses the object. Encapsulate the complex object in a simple bounding box, and use the simple bounding box shape to approximate the shape of the complex geometric body.

[0061] In an embodiment of the present invention, the deep learning-based target detection algorithm may be a Single Shot MultiBox Detector (SSD) algorithm or a Yolo (You Only Look Once) target detection algorithm.

[0062] Among them, the Yolo target detection algorithm is a new target detection method developed based on the Convolutional Neural Network (CNN). It integrates target area prediction and target category prediction into a single neural network model to achieve rapid target detection and recognition with high accuracy.

[0063] Specifically, the Yolo method combines target area prediction and target category prediction into one, using a single convolutional neural network that uses information from the entire image to predict the target’s bounding box and identify the target’s category, achieving end-to-end target detection with fast detection speed and real-time detection.

[0064] Among them, the SSD algorithm is an object detection algorithm that directly predicts the coordinates and categories of the target box. The SSD algorithm uses the 16-layer deep convolutional network VGG-16 as the basic network, changes two fully connected layers into convolutional layers, and adds 4 convolutional layers to construct the network structure. The SSD algorithm uses the output of multiple convolutional layers for classification and position regression at the same time, maintaining a high recognition speed.

[0065] In an embodiment of the present invention, after performing target detection on an image of settlement supplies using a deep learning-based target detection algorithm, the bounding box of each settlement supply in the image and the category corresponding to the object in each bounding box can be output.

[0066] Taking the application scenario of a fast food restaurant as an example, the settlement items are plates. After obtaining the image information of multiple plates carrying the goods to be settled, the target detection algorithm based on deep learning is used to detect the image including multiple plates to obtain the bounding box of each plate.

[0067] Among them, the target detection algorithm based on SSD or Yolo has the advantages of higher accuracy and better robustness. Therefore, by using the target detection algorithm based on SSD or Yolo to perform target detection on the image information of settlement supplies, the settlement supplies included in the image can be accurately identified, thereby improving the accuracy of settlement. In addition, the recognition speed of the target detection algorithm based on SSD or Yolo is high, so the speed of settlement can be greatly improved.

[0068] S1102, based on the pre-trained convolutional neural network, obtain the pre-set settlement mark on the settlement supplies from the bounding box, and extract the color coding information of the settlement mark.

[0069] In an embodiment of the present invention, before using the settlement mark of the settlement supplies for settlement, it is necessary to pre-set the settlement mark on the settlement supplies. The settlement mark is used to represent the color coding information on the settlement supplies. The settlement mark includes at least two colors, which is used to increase the types of identification information to improve the accuracy of identification. In other words, the color coding information on the carrier is represented by at least two colors.

[0070] In some examples, before acquiring the image information of the checkout supplies, the following steps are included:

[0071] A, extract multiple colors from multiple categories of goods.

[0072] In the embodiment of the present invention, before setting the settlement mark, it is necessary to extract the color used in the settlement mark. In order to ensure that the color in the settlement mark is more natural and real and improve the user's good experience, the color in the settlement mark is the true color of the commodity extracted from a variety of commodities.

[0073] As an example, the settlement items are dinner plates, and the colors in the settlement logo on the dinner plates can be the natural colors extracted from fruits and vegetables. Figure 2 First, the food colors of various foods are extracted from vegetables and fruits. The categories of the extracted food colors can be greater than 100. The colors extracted from fruits and vegetables are more natural and real, and will not affect the visual senses of diners, so they will not cause diners to lose appetite.

[0074] B. Use the principle of maximizing color space distance to select the color used in the settlement mark from a variety of colors.

[0075] Continue to see Figure 2 , we can use the principle of maximizing spatial distance to extract N natural colors from a variety of food colors and use them as the colors used in settlement identification. Color space distance refers to the difference between colors. Generally, the greater the color space distance between two colors, the greater the difference between the two colors. The greater the difference between two colors, the easier it is to identify the colors.

[0076] As a specific example, in the RGB space, the calculation formula for the distance between two colors is similar to the calculation formula for the Euclidean distance between two points.

[0077] As an example, 10 colors can be identified from multiple types of fruits and vegetables in advance, and then 4 colors can be selected from the 10 colors as the colors used in the settlement information identification area by using the color space distance maximization principle. Specifically, the distance between every two colors in the 10 colors, that is, 45 distances, is first calculated. Then, based on the calculated 45 distances, 4 colors with the largest distance difference are selected.

[0078] In the embodiment of the present invention, by extracting N true colors according to the principle of maximizing the color space distance, different colors in the settlement mark can be made more recognizable, thereby improving the recognition accuracy.

[0079] C. Set the settlement mark using the color used in the selected settlement mark.

[0080] In an embodiment of the present invention, after selecting a plurality of colors used in a settlement mark, the plurality of colors are combined and classified, and a settlement mark is set on a settlement item using the allocated plurality of color combinations. Each color combination corresponds to different color coding information. Each color combination is used to mark the category of the settlement item and to identify the settlement information of the commodity carried on the settlement item. The settlement marks with different color combinations correspond to different settlement information. In other words, the color coding information represented by different settlement marks corresponds to different settlement information.

[0081] In some examples, the settlement information may be price information of the product or other information used to indicate the value of the product.

[0082] As an example, see Figure 2 , after extracting N colors for setting the settlement mark from the various colors of fruits and vegetables using the principle of maximizing the color space distance, any two colors among the N colors are combined in pairs to form multiple color combinations. The color information of the settlement mark of the dinner plate can include two colors in any color combination. Among them, the two colors included in the settlement mark of the dinner plate are used to identify the settlement information of the food carried on the dinner plate.

[0083] See also Figure 2 , the selected N colors are combined in pairs to generate multiple color combinations. Depending on the number of N, 16, 32 or 64 color combinations can be generated. After the color combination for setting the settlement mark is determined, the settlement mark represented by the above color combination is printed on the settlement supplies.

[0084] As an example, Figure 2 Four different settlement logos are shown printed on a dinner plate. Each settlement logo includes two color rings of different colors. For the upper left corner, the color of the outer color ring is pink, and the color of the inner color ring is dark red. For the upper right corner, the color of the outer color ring is green, and the color of the inner color ring is red. For the lower left corner, the color of the outer color ring is green, and the color of the inner color ring is yellow. For the lower right corner, the color of the outer color ring is yellow, and the color of the inner color ring is black.

[0085] However, if only the information on the type and number of colors in the settlement mark is used without considering the arrangement information of the colors, the number of color combinations used to represent the settlement information will be reduced.

[0086] For example, the settlement mark printed on the plate is two color rings set on the edge of the circular plate. In other words, the color coding information of the plate is represented by two-color annular color bands arranged along the edge of the plate. Among them, in the extension direction from the periphery to the center of the plate, the colors of the two color rings are red and yellow. If the color arrangement information is not considered, as long as the settlement mark includes red and yellow, the corresponding settlement information is the same.

[0087] In order to obtain more color combinations that can identify settlement information, the color information can be sorted. For example, in the extension direction from the periphery to the center of the plate, red and yellow and yellow and red correspond to different settlement information. That is, two colors can form at least two settlement marks, each settlement mark corresponds to unique color coding information, that is, each settlement mark corresponds to unique price information.

[0088] That is, even if the color types in the settlement mark are the same, if the arrangement of the colors along the predetermined direction is different, the corresponding color coding information is different, that is, the corresponding settlement information is also different. As a specific example, if the color information of the settlement mark on the plate is red and yellow, the price of the food on the plate is 10 yuan, and if the color information of the settlement mark on the plate is red and blue, the price of the food on the plate is 8 yuan.

[0089] Optionally, the settlement mark on the settlement supplies may also include three colors or more than three colors.

[0090] In some examples, the settlement mark includes a plurality of color sub-regions arranged at intervals, each color sub-region includes a color, wherein at least two color sub-regions include different colors.

[0091] As another example, if the pre-assigned color combination includes two color combinations of red and green and red and blue. For example, the settlement mark can be three color rings spaced apart on the edge of a circular dinner plate. The three color rings include a red color ring and two green color rings. For another example, the three color rings of the settlement mark include a red color ring and two blue color rings. In this example, at least two color sub-areas in the settlement mark include different colors.

[0092] In some examples, by arranging multiple color sub-regions at intervals, color interference between two adjacent color sub-regions can be avoided, thereby improving the accuracy of color recognition.

[0093] In some examples, in order to further improve the accuracy of identifying the color information in the settlement mark, the color in each color sub-region may be different.

[0094] As an example, if the pre-assigned color combinations include red, green and blue and red, yellow and blue. For example, the settlement mark can be three annular color rings spaced apart on the edge of a circular dinner plate. The settlement mark includes a red color ring, a green color ring and a blue color ring. For another example, the settlement mark includes a red color ring, a yellow color ring and a blue color ring.

[0095] In some examples, when the colors in each color sub-region are different, the interval between every two adjacent color sub-regions may be approximately 0, so as to reduce the area occupied by the settlement logo and avoid giving the user a feeling of clutter.

[0096] As a specific example, if the settlement item is a dinner plate, the color sub-area may be an annular color band extending along the outer circumference of the dinner plate. And each color sub-area includes different colors. For example, multiple annular color bands may be provided on the outer surface of the dinner plate, and the color of each color band is different.

[0097] Preferably, color rings of two or more colors can be set on the edge of the plate. By printing color rings of multiple colors on the edge of the plate, the influence of various lighting and occlusion can be effectively overcome. Because only a small section of the color ring is needed to effectively identify multiple color information in the settlement mark, the efficiency of recognition is improved, and the robustness is high.

[0098] In some examples, a checkout logo may be printed on the checkout supplies.

[0099] In the embodiment of the present invention, after placing the commodity on the settlement article corresponding to the settlement information of the commodity, the commodity can be settled using the color coding information represented by the settlement mark on the settlement article.

[0100] In some examples, the pre-trained convolutional neural network is a network that can perform semantic segmentation and object detection at the same time. Object detection refers to finding the rectangular box containing a class of objects for a given image. Semantic segmentation is to classify each pixel in the image into a corresponding category, that is, to classify an image at the pixel level.

[0101] As a specific example, the convolutional neural network pre-trained using the sample image is the Mask R-CNN convolutional neural network. The Mask R-CNN convolutional neural network can complete target detection, classify each target, and achieve high-quality pixel-level target segmentation in each target.

[0102] Since the Mask R-CNN convolutional neural network can achieve high-quality semantic segmentation, the Mask R-CNN convolutional neural network can be used to extract the settlement mark inside each bounding box and accurately extract the color information in the settlement mark.

[0103] In the embodiment of the present invention, the color information of the settlement information identification area on each plate is extracted with better accuracy based on the Mask R-CNN convolutional neural network, and is not easily affected by the dishes and the table background. In addition, only a relatively small number of pixels are needed to extract the color information of the settlement mark, so even if the plate is relatively large, the color information of the settlement mark can be accurately identified, and the robustness is strong.

[0104] In an embodiment of the present invention, settlement supplies detection and color extraction in settlement marks are performed using a deep learning-based method, with better detection effect and higher extraction accuracy. At the same time, only relatively few pixels are needed to identify the color in the settlement mark, which is not affected by occlusion and has good robustness.

[0105] As a specific example, see Figure 2 If the settlement items are plates, when the user places multiple plates with items to be settled into the designated detection area, the camera will obtain the plate image. After obtaining the plate image from the camera, the bounding box of each plate is obtained based on the deep learning target detection algorithm, and then the settlement mark area in each bounding box is extracted using the deep learning algorithm, and the color information of the settlement mark on each plate is extracted using the deep learning algorithm.

[0106] Then, according to the preset correspondence between the color coding information and the color information, the color coding information corresponding to the color information in the settlement mark can be obtained. The color information in the settlement mark can be pre-coded to obtain the preset correspondence between the color information and the color coding information.

[0107] As an example, in the correspondence between the preset color information and the color coding information, in the extension direction from the periphery to the center of the plate, the color codes corresponding to the three different color information of red, yellow and blue, red, blue and yellow, and blue and red are 1, 2 and 3 respectively. Different color coding information corresponds to different settlement information.

[0108] In an embodiment of the present invention, based on the correspondence between preset color coding information and settlement information, the settlement information corresponding to the color coding information represented by each settlement mark is determined, and the settlement information corresponding to the color coding information is used as the settlement information of the goods carried on the plate where the settlement mark is located.

[0109] In the above example, if the color information in the extension direction from the periphery to the center of the plate in the settlement mark of the settlement supplies is red, yellow, and blue, then the color code corresponding to red, yellow, and blue can be obtained as 1. According to the preset correspondence between the color code information and the settlement information, the settlement information corresponding to the color code 1 can be obtained. Among them, the settlement information corresponding to the color code information is the settlement information of the goods to be settled on the settlement supplies.

[0110] According to the color coding information represented by the settlement mark on each plate, the price information of the goods to be settled on each plate can be obtained to calculate the total price of this settlement.

[0111] In some embodiments of the present invention, the color information in the extracted settlement mark can be directly used to directly obtain the settlement information of the goods to be settled carried on the settlement supplies for the settlement of the goods.

[0112] Specifically, according to the preset correspondence between the color information and the settlement information and the extracted color information in the settlement mark, the settlement information corresponding to the color information in the settlement mark is obtained. Then, the settlement information corresponding to the color information in the settlement mark is used as the settlement information of the commodity to be settled, and the commodity to be settled is settled based on the settlement information of the commodity to be settled.

[0113] The commodity settlement method provided by the embodiment of the present invention can realize accurate identification and self-service settlement by printing a customized settlement mark representing color coding information on the settlement supplies. The settlement supplies do not need additional modification, and have the characteristics of low cost, fast identification speed and high accuracy. In addition, the settlement supplies that need to be washed can be washed repeatedly, thereby extending the use time of the settlement supplies.

[0114] Figure 3 The schematic diagram of the structure of a commodity settlement system provided according to an embodiment of the present invention is shown. The commodity settlement system provided by the embodiment of the present invention comprises: a carrier 310 with color coding information, a camera 320 and a settlement information processing device 330, wherein:

[0115] The camera 320 is used to obtain image information for settlement, and the image information includes one or more carriers 310 with color coding information.

[0116] The carrier 310 with color coding information is used to accommodate the goods to be settled. The color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance.

[0117] The settlement information processing device 330 is used to determine the settlement information corresponding to each carrier according to the color coding information on each carrier in the image information, for commodity settlement.

[0118] In an embodiment of the present invention, the settlement information processing device 330 processes the image information through a target detection algorithm based on deep learning to obtain a bounding box for each carrier in the image information, and extracts color coding information within each bounding box through a deep learning algorithm.

[0119] In an embodiment of the present invention, the deep learning-based target detection algorithm includes a single shot multi-box detection SSD algorithm and / or a Yolo algorithm.

[0120] In an embodiment of the present invention, at least two colors are extracted from a plurality of colors, and when extracting, all colors are subjected to contrast processing, and at least two colors are extracted using the principle of maximizing the color space distance. In an embodiment of the present invention, at least two colors include colors extracted from vegetables and / or fruits.

[0121] In an embodiment of the present invention, the color coding information corresponds one-to-one to the price information.

[0122] In an embodiment of the present invention, the carrier is a dinner plate, and the color coding information is represented by an annular color band of two colors arranged along the edge of the dinner plate.

[0123] Other details of the commodity settlement system according to the embodiment of the present invention are combined with the above Figure 1 to Figure 2 The method described in the embodiment of the present invention is similar and will not be described in detail here.

[0124] The commodity settlement system provided by the embodiment of the present invention provides more identification information and improves the accuracy of self-service settlement by using more than two colors to mark the settlement information of commodities.

[0125] Figure 4 FIG. 2 shows a schematic diagram of the structure of a commodity settlement device provided according to an embodiment of the present invention. Figure 4 As shown, the commodity settlement device 400 includes:

[0126] The acquisition module 410 is used to acquire image information for settlement, wherein the image information includes one or more carriers with color coding information, wherein the carrier with color coding information is used to accommodate goods to be settled, and the color coding information is represented by at least two colors, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance.

[0127] The settlement module 420 is used to determine the settlement information corresponding to each carrier according to the color coding information on each carrier in the image information, for commodity settlement.

[0128] The commodity settlement system provided by the embodiment of the present invention determines the settlement information of the commodity to be settled by using the settlement mark including at least two colors pre-set on the settlement supplies. Compared with a single color, it has more identification information and is not easily affected by the color of the commodity to be settled itself during settlement, thereby improving the accuracy of self-service settlement.

[0129] Other details of the commodity settlement device according to the embodiment of the present invention are combined with the above Figure 1 to Figure 2 The method described in the embodiment of the present invention is similar and will not be described in detail here.

[0130] Figure 5 A flowchart of a commodity identification method 500 provided in an embodiment of the present invention is shown. Figure 5 As shown, the commodity identification method provided by the embodiment of the present invention includes the following steps:

[0131] S510, obtaining image information for identification, wherein the image information includes one or more commodity placement areas with color coding information, wherein the commodity placement areas with color coding information are used to place commodities, the color coding information is represented by at least two colors, and a distance between any two colors of the at least two colors in a color space is greater than a predetermined distance.

[0132] S520, determining commodity information corresponding to each commodity placement area according to the color coding information of each commodity placement area in the image information for commodity identification.

[0133] The commodity identification method provided by the embodiment of the present invention determines commodity information by using color coding information represented by at least two colors on the commodity placement area. Compared with a single color, it has more identification information and improves the accuracy of commodity identification.

[0134] The specific implementation process of the commodity identification method of the embodiment of the present invention is introduced below using the application scenario of supermarket shelf arrangement. There are designated commodity placement areas on the commodity shelves of the supermarket for placing different commodities. As the commodities are sold, some commodity placement areas will become vacant. When a commodity placement area becomes vacant, the supermarket manager needs to replenish the corresponding commodities to the vacant commodity placement area in a timely manner.

[0135] By utilizing the commodity identification method provided by the embodiment of the present invention, the types of commodities that should be placed in the vacant commodity placement area can be automatically identified, and then the supermarket manager is notified to replenish the corresponding types of commodities.

[0136] As an example, there are 10 D product placement areas on the shelf for placing braised instant noodles, and 5 F product placement areas for placing spicy instant noodles. The color coding information on the product placement areas of the same product is the same. That is, the color coding information of the 10 D product placement areas is the same, and the color coding information of the 5 F product placement areas is the same.

[0137] Similar to the above commodity settlement method, an identification mark representing color coding information can be printed in advance in the commodity placement area, wherein each identification mark includes at least two colors and each identification mark has corresponding color coding information.

[0138] Similar to the above commodity settlement method, the color in the commodity placement area is extracted from multiple colors. When extracting, all colors are compared and processed, and multiple colors that are easy to distinguish are extracted using the principle of maximizing the color space distance, which will not be repeated here.

[0139] In an embodiment of the present invention, the color coding information of each commodity placement area corresponds to the commodity information placed on the commodity placement area. The commodity information may include the commodity type and the quantity of the commodity of the type.

[0140] As a specific example, the identification mark of the product placement area is two color rings arranged along the edge of the product placement area. For example, the edge of the D product placement area for braised instant noodles is printed with a red ring and a yellow ring from the inside to the outside, while the edge of the F product placement area for spicy instant noodles is printed with a black ring and a green ring from the inside to the outside.

[0141] When corresponding products are not placed in one or more product placement areas, image information including one or more product placement areas with color coding information but without products placed therein can be obtained. As an example, if there are 2 D product placement areas and 2 F product placement areas without products placed therein, image information of these 4 product placement areas can be obtained.

[0142] Then, by processing the image information using a target detection algorithm based on deep learning, the bounding box of each product placement area in the image information can be obtained. Then, the color information in each bounding box is extracted based on the deep learning algorithm. Among them, the target detection algorithm based on deep learning can be an SSD target detection algorithm or a Yolo target detection algorithm.

[0143] Among them, the color information corresponding to the D product placement area is red and yellow arranged from the inside to the outside, and the color information corresponding to the F product placement area is black and green arranged from the inside to the outside.

[0144] Next, according to the correspondence between the pre-stored color information and the color coding information, the first color coding corresponding to red and yellow and the second color coding corresponding to black and green can be obtained.

[0145] According to the correspondence between the color coding information and the product information, the product types corresponding to the two D product placement areas and the product types corresponding to the two F product placement areas can be determined.

[0146] After the commodity types corresponding to the commodity placement areas are identified, the commodity types corresponding to different commodity placement areas can be sent to supermarket managers, thereby reducing manual involvement and improving sorting efficiency.

[0147] The commodity identification method provided by the embodiment of the present invention is also applicable to the application scenario of inventory in a supermarket. In this application scenario, the commodity placement area is a warehouse. Each commodity placement area can be placed with a predetermined number of commodities of the same type. The commodity information corresponding to the color coding information of each commodity placement area is the commodity type and the quantity corresponding to the commodity of the type.

[0148] By identifying the color-coded information of the merchandise placement areas where no merchandise is placed, it can be determined what type of merchandise is missing in the warehouse and the quantity of that merchandise, thereby helping supermarket managers to conduct automatic inventory.

[0149] Combination Figure 1 , Figure 2 and Figure 4 The commodity settlement method and apparatus described in the embodiments of the present invention may be implemented by a commodity settlement device. Figure 6 FIG. 6 is a schematic diagram showing a hardware structure 600 of a commodity settlement device according to an embodiment of the invention.

[0150] like Figure 6As shown, the commodity settlement device 600 includes an input device 601, an input interface 602, a central processing unit 603, a memory 604, an output interface 605, and an output device 606. The input interface 602, the central processing unit 603, the memory 604, and the output interface 605 are connected to each other through a bus 610, and the input device 601 and the output device 606 are connected to the bus 610 through the input interface 602 and the output interface 605, respectively, and then connected to other components of the commodity settlement device 600.

[0151] Specifically, the input device 601 receives input information from the outside and transmits the input information to the central processing unit 603 through the input interface 602; the central processing unit 603 processes the input information based on the computer executable instructions stored in the memory 604 to generate output information, temporarily or permanently stores the output information in the memory 604, and then transmits the output information to the output device 606 through the output interface 605; the output device 606 outputs the output information to the outside of the commodity settlement device 600 for user use.

[0152] That is to say, Figure 6 The commodity settlement device shown can also be implemented as comprising: a memory storing computer executable instructions; and a processor, which can realize the combination of Figure 1 , Figure 2 and Figure 3 The commodity settlement method and device described.

[0153] An embodiment of the present invention further provides a computer storage medium having computer program instructions stored thereon; when the computer program instructions are executed by a processor, the commodity settlement method provided by the embodiment of the present invention is implemented.

[0154] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0155] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0156] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A commodity settlement method, comprising: Acquire image information for settlement, wherein the image information includes one or more settlement supplies with color coding information, wherein the settlement supplies with color coding information are used to accommodate commodities to be settled, the color coding information is represented by at least two colors, the at least two colors are the true colors of commodities extracted from the commodities to be settled, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance; the color coding information on the settlement supplies is represented by a settlement mark pre-set on the settlement supplies, the settlement mark is represented by a color combination for setting the settlement mark, and the number of the color combinations for setting the settlement mark is determined according to the type and number of colors for setting the settlement mark and the arrangement information of the colors included in the color combination along a predetermined direction; According to the color coding information on each of the settlement supplies in the image information, the settlement information corresponding to each settlement supply is determined respectively for commodity settlement.

2. The method according to claim 1, characterized in that The image information is processed by a target detection algorithm based on deep learning to obtain a bounding box of each settlement item in the image information, and the color coding information in each bounding box is extracted by a deep learning algorithm.

3. The method according to claim 2, characterized in that The deep learning-based target detection algorithm includes a single shot multi-box detection (SSD) algorithm and / or a Yolo algorithm.

4. The method according to claim 1, characterized in that The at least two colors are extracted from the commodities to be settled. When extracting, all colors are compared and processed, and the at least two colors are extracted by using the principle of maximizing the color space distance.

5. The method according to claim 1, characterized in that The at least two colors include colors extracted from vegetables and / or fruits.

6. The method according to claim 1, characterized in that The color coding information corresponds to the price information one by one.

7. The method according to any one of claims 1 to 6, characterized in that The settlement item is a dinner plate, and the color coding information is represented by an annular color band of two colors arranged along the edge of the dinner plate.

8. A commodity settlement system, comprising: A camera, a settlement item with color-coded information, and a settlement information processing device, wherein: The camera is used to obtain image information for settlement, wherein the image information includes one or more settlement supplies with color coding information; The settlement supplies with color coding information are used to accommodate commodities to be settled, the color coding information is represented by at least two colors, the at least two colors are the true colors of commodities extracted from the commodities to be settled, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance; the color coding information on the settlement supplies is represented by a settlement mark pre-set on the settlement supplies, the settlement mark is represented by a color combination for setting the settlement mark, and the number of the color combinations for setting the settlement mark is determined according to the type and number of colors for setting the settlement mark and the arrangement information of the colors included in the color combination along a predetermined direction; The settlement information processing device is used to determine the settlement information corresponding to each settlement item according to the color coding information on each settlement item in the image information, for commodity settlement.

9. A commodity settlement device, characterized in that: The device comprises: An acquisition module is used to acquire image information for settlement, wherein the image information includes one or more settlement supplies with color coding information, wherein the settlement supplies with color coding information are used to accommodate commodities to be settled, the color coding information is represented by at least two colors, the at least two colors are the true colors of commodities extracted from the commodities to be settled, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance; the color coding information on the settlement supplies is represented by a settlement mark pre-set on the settlement supplies, the settlement mark is represented by a color combination for setting the settlement mark, and the number of the color combinations for setting the settlement mark is determined according to the type and number of colors for setting the settlement mark and the arrangement information of the colors included in the color combination along a predetermined direction; The settlement module is used to determine the settlement information corresponding to each settlement item according to the color coding information on each settlement item in the image information, for commodity settlement.

10. A carrier with color-coded information, characterized in that: The carrier is used to accommodate goods to be settled, and the carrier is a settlement item, and the settlement item is used to accommodate goods to be settled. The color coding information is represented by at least two colors, and the at least two colors are the original colors of the goods extracted from the goods to be settled. The distance between any two colors of the at least two colors in the color space is greater than a predetermined distance; the color coding information on the settlement item is represented by a settlement mark pre-set on the settlement item, and the settlement mark is represented by a color combination for setting the settlement mark, and the number of color combinations for setting the settlement mark is determined according to the type and number of colors used to set the settlement mark and the arrangement information of the colors included in the color combination along a predetermined direction.

11. The carrier according to claim 10, characterized in that The at least two colors are extracted from the commodities to be settled. When extracting, all colors are compared and processed, and the at least two colors are extracted by using the principle of maximizing the color space distance.

12. The carrier according to claim 10, characterized in that The carrier is a dinner plate, and the color-coded information is represented by an annular color band of two colors arranged along the edge of the dinner plate.

13. A commodity identification method, characterized in that: The method comprises: Acquire image information for identification, wherein the image information includes one or more commodity placement areas with color coding information, wherein the commodity placement areas with color coding information are used to place commodities, the color coding information is represented by at least two colors, the at least two colors are the true colors of commodities extracted from commodities to be settled, and the distance between any two colors of the at least two colors in the color space is greater than a predetermined distance; the commodity placement area is preset with an identification mark for representing the color coding information, the identification mark includes the corresponding color coding information; the identification mark is represented by a color combination for setting the identification mark, and the number of the color combinations for setting the identification mark is determined according to the type and number of colors for setting the identification mark and the arrangement information of the colors included in the color combination along a predetermined direction; According to the color coding information of the identification mark of each commodity placement area in the image information, commodity information corresponding to each commodity placement area is determined respectively for commodity identification.

14. The method according to claim 13, characterized in that The image information is processed by a deep learning-based target detection algorithm to obtain a bounding box of each commodity placement area in the image information, and the color coding information in each bounding box is extracted by a deep learning algorithm.

15. The method according to claim 14, characterized in that The deep learning-based target detection algorithm includes an SSD algorithm and / or a Yolo algorithm.

16. The method according to claim 13, characterized in that The at least two colors are extracted from the commodities to be settled. When extracting, all colors are compared and processed, and the at least two colors are extracted by using the principle of maximizing the color space distance.

17. The method according to claim 13, characterized in that The color coding information corresponds to the commodity information one by one.

18. The method according to any one of claims 13 to 17, characterized in that: The one or more commodity placement areas belong to commodity racks or warehouses, and the color coding information is represented by an annular color band of two colors arranged along the edge of the commodity placement area.

19. A computing device, characterized in that The computing device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the commodity settlement method according to any one of claims 1 to 7 is implemented.

20. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the commodity settlement method according to any one of claims 1 to 7 is implemented.

Images