Intelligent identification and settlement system and method for multiple commodities
Through multiple image sensors and weighing units combined with intelligent control units, the identification problems of multiple products under occlusion and the cumbersome new product entry problems are solved, high-precision identification and settlement are achieved, and operation costs and customer waiting time are reduced. , promote the intelligent upgrade of the retail industry.
Patent Information
- Application Number
- CN202411094503.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-10
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing intelligent identification and settlement system for multi-item products has shortcomings in accuracy, automation and customer comfort, especially in the case of occlusion, and the identification and entry process of new products is complicated, resulting in high operating costs for merchants and customers. Long wait time.
Using multiple image sensors and weighing units, combined with intelligent control units and product information input units, the accurate identification and settlement of multiple products is achieved through image recognition, weighing and interchange recognition technology, and customers are allowed to scan the code. The device updates the product database, simplifies the settlement process and identifies new products.
Improve the identification accuracy and settlement accuracy of multiple products, reduce merchant operating costs, shorten customer waiting time, enhance system adaptability, support rapid identification and settlement of new products, and promote the development of the retail industry to automation and intelligence. .
Smart Images

Figure CN118968681A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to intelligent identification and settlement of commodities, and in particular to an intelligent identification and settlement system and method for multiple commodities. Background Art
[0002] At present, with the development of artificial intelligence, big data and other technologies in the traditional retail industry, some intelligent pricing systems have been seen in supermarkets, convenience stores, bakeries and canteens. Each system has different focuses, mainly involving single standard product pricing, multi-standard product joint pricing and single non-standard product weighing pricing. This invention focuses on multi-standard product joint pricing.
[0003] The current application of the technology of the intelligent pricing system is in a scenario such as a smart canteen. For the canteen meal scene, the merchant pre-fixes different foods on small plates. When customers take different plates and put them on the same tray, the system will identify different items and then self-service identification and settlement. Among them, the hardware part of this technology consists of a camera placed on the top floor and a fixed settlement platform. The software part of the system includes two parts: background training and front-end recognition. In the background training stage, the system needs to know the categories of items that can be stored in the canteen in advance, train a specified number of item classification models, and thus build a fixed number of classifiers; in the front-end recognition stage, object detection technology is used to obtain the bounding boxes of different small plates, and self-service settlement is performed according to the number of items and the price of the plates.
[0004] Another application scenario of the technology currently used in the intelligent pricing system is a self-service bakery: for the self-service purchasing scenario of the bakery, customers select different breads and store them on the same tray. After laying the items flat, the system recognizes the different items and then performs self-service identification and settlement. The hardware part consists of a top-level camera, a fixed settlement platform, and an on-end reasoning and recognition module. The software part of the system includes two parts: background training and front-end recognition. In the background training stage, the categories of items that the bakery can store are fixed, a classification model for a specified number of items is pre-trained, and a fixed number of classifiers are constructed; in the front-end recognition stage, object detection technology is used to obtain the bounding boxes of different breads, and self-service settlement is performed according to the number of items and the price of the plate.
[0005] These technologies or patents still have room for improvement in terms of accuracy, automation or customer comfort. For the above two application scenarios, since the obscured items cannot be seen within the visual range of a camera, the food needs to be laid out flat. In some special scenarios, if the customer cheats and deliberately stacks the items for pricing, it will cause losses to the merchant; therefore, this technology lacks accuracy and is difficult to handle multi-standard product recognition under occlusion.
[0006] In response to the occlusion detection problem, Chinese Patent 1 [CN201810686023] designed an end-to-end method for determining the type and quantity of objects under multiple cameras. It sets up multiple cameras and builds an end-to-end deep neural network. The input is multi-camera features and the output is the type and quantity of items. This method does not obtain the position of the bounding box, making it difficult for humans to analyze and verify. Chinese Patent 2 [CN201810964593] collects and obtains the total 3D volume of items in the shopping cart, and obtains the number of items through the volume of individual items and the total volume. Although the 3D scene is collected from multiple angles and the total volume and individual item volume are used to determine the number of items, this method is difficult to adapt to the occlusion of multiple individual items of different sizes because there are gaps and superposition compression between such individual items. In addition, for non-rigid objects such as bread, the accuracy of 3D volume acquisition is insufficient.
[0007] In addition, for new products (product information that has not been entered into the database), the existing system requires merchants to enter information and requires that the categories of pre-stored items in the warehouse are fixed. If a new product appears, the clerk needs to perform a series of warehousing operations, that is, collect a certain number of pictures of the new product to update the background model, which will bring great inconvenience. On the one hand, the warehousing operation is time-consuming and labor-intensive, and if new products are launched frequently, it will bring extra work to merchants; on the other hand, if a new product is missed in the warehouse, when the user uses it, he cannot check out in time, causing user dissatisfaction.
[0008] It can be seen that the above-mentioned existing intelligent identification and settlement system for multiple commodities still has inconveniences and defects in method and use, and urgently needs to be further improved. Summary of the invention
[0009] The main purpose of the present invention is to overcome the defects of the existing intelligent identification and settlement system for multiple commodities and to provide a new intelligent identification and settlement system for multiple commodities. The technical problem to be solved is to improve the identification accuracy and settlement accuracy of multiple commodities. By optimizing the recognition algorithm and introducing stacking / occlusion processing technology, the recognition problem in the case of commodity stacking and occlusion is effectively solved, thereby being more practical and having industrial utilization value.
[0010] Another object of the present invention is to provide an intelligent identification and settlement system for multiple commodities. The technical problem to be solved is to assist merchants in updating commodity databases through customers' purchasing behavior, thereby reducing merchants' operating costs.
[0011] Another object of the present invention is to provide an intelligent identification and settlement system for multiple commodities. The technical problem to be solved is to simplify the settlement process, shorten the customer waiting time, and enhance the customer shopping experience, thereby making it more practical.
[0012] Another object of the present invention is to provide an intelligent identification and settlement system for multiple commodities. The technical problem to be solved is to enhance the adaptability of the system so that the system supports rapid identification and settlement of newly listed commodities, meets the needs of merchants for rapid iteration of commodities, and is thus more suitable for practical use.
[0013] Another object of the present invention is to provide an intelligent identification and settlement system for multiple commodities to promote the intelligent upgrading of the retail industry: to promote the development of the retail industry towards automation and intelligence, and to improve overall operational efficiency and service levels.
[0014] The purpose of the present invention and the technical problem solved are achieved by adopting the following technical solutions. According to a smart identification and settlement system for multiple commodities proposed by the present invention, the system includes an identification unit, a settlement unit and a storage unit; the identification unit generates a signal after identifying the multiple commodities and transmits the signal to the settlement unit, and the settlement unit calls the data of the storage unit to complete the identification and settlement of the multiple commodities.
[0015] The purpose of the present invention and the solution to its technical problems can be further achieved by adopting the following technical measures.
[0016] In the aforementioned intelligent identification and settlement system for multiple commodities, the identification unit includes one or more image sensors for detecting and identifying the name and quantity of the commodities.
[0017] The aforementioned intelligent identification and settlement system for multiple commodities, wherein accurate identification of multiple commodities is achieved by designing and arranging the number and positions of the image sensors.
[0018] In the aforementioned intelligent identification and settlement system for multiple commodities, the design and arrangement of the number and positions of the image sensors can realize the identification of the names and quantities of the stacked commodities.
[0019] In the aforementioned intelligent identification and settlement system for multiple commodities, the settlement unit includes a weighing unit to achieve weighing of the weights of the multiple commodities that need to be identified.
[0020] The aforementioned intelligent identification and settlement system for multiple commodities, wherein the data stored in the storage unit is used for identification of commodities and settlement of commodity prices, and the data includes commodity name, weight, price, code and outer packaging image of the commodity.
[0021] The aforementioned intelligent identification and settlement system for multiple commodities, wherein the system further comprises an intelligent control unit, which retrieves the commodity information in the storage unit according to the identification result of the identification unit, and compares the weight with the multiple commodities obtained by the settlement unit, thereby realizing accurate identification and settlement of multiple commodities.
[0022] The aforementioned intelligent identification and settlement system for multiple items, wherein the system further comprises a product information input unit, wherein the product information input unit is used for a user to scan the coding information of a product that is not identified or is incorrectly identified by the identification system, and the intelligent control unit inputs the information obtained after the user scans into the storage unit, thereby realizing the identification of the unidentified or incorrectly identified products, the entry and calculation of information, and then used for the subsequent identification and settlement of the product by the identification system.
[0023] In the aforementioned intelligent identification and settlement system for multiple commodities, the commodity information input unit is a barcode scanner or a key input device or a similar input device.
[0024] The aforementioned intelligent identification and settlement system for multiple commodities comprises:
[0025] an identification unit, the identification unit comprising one or more image sensors and a display;
[0026] A settlement unit, the settlement unit comprising a weighing unit and a calculation unit;
[0027] A storage unit, wherein the data stored in the storage unit includes the name, weight, price, code and outer packaging image of the commodity;
[0028] An intelligent control unit, which is capable of calling data from the storage unit, the identification unit, and the settlement unit;
[0029] A commodity information input unit, wherein the commodity information input unit is used for a user to input information of an unrecognized or incorrectly recognized commodity;
[0030] in:
[0031] The data acquired by the identification unit is used for settlement by the settlement unit and comparative analysis by the intelligent control unit, so as to finally determine the name, quantity and price of the commodity;
[0032] The weighing unit in the settlement unit obtains the weight data of the commodity and transmits it to the intelligent control unit, so that the intelligent control unit compares the quantity and weight of multiple commodities to determine whether the quantity and name of the commodities to be settled are correct; and
[0033] The product information input unit prompts the user to enter the product information based on the information feedback from the intelligent control unit when the product purchased by the user cannot be identified or is identified incorrectly, and stores the information in the storage unit for subsequent identification and settlement of the product by the system.
[0034] In the aforementioned intelligent identification and settlement system for multiple commodities, the identification unit comprises a quantity identification unit, a stacking identification unit and a commodity identification unit.
[0035] The purpose of the present invention and the solution to its technical problems are also achieved by adopting the following technical solutions. According to the present invention, a method for intelligent identification and settlement of multiple commodities includes the following steps:
[0036] S1: The user places the goods to be settled on the settlement counter for intelligent identification. According to the intelligent identification results, the following steps are performed respectively;
[0037] S2: If the goods to be settled are identified in step S1 to be severely blocked, so that the goods to be settled cannot be accurately identified, or the quantity and weight of the identified goods are greatly different from the goods information in the storage unit, the user is prompted to rearrange the goods, and then step S1 is repeated to complete the identification of multiple goods;
[0038] S3: If it is found in step S1 or step S2 that there are unrecognizable or misrecognized commodities, the user is prompted to scan the unrecognized or misrecognized commodities with a barcode scanner, and the unrecognized or misrecognized commodities are corrected, and the system completes data entry at the same time;
[0039] S4: The intelligent identification unit determines whether all commodities on the checkout counter are correctly identified and the weight verification is passed, calculates the unit price, quantity, amount, etc. of all commodities, and displays the settlement list.
[0040] The purpose of the present invention and the solution to its technical problems can be further achieved by adopting the following technical measures.
[0041] The aforementioned intelligent identification and settlement system for multiple items, wherein in step S3, the information of unidentified or incorrectly identified items scanned by the user is stored in the storage unit, thereby realizing the entry and storage of the unidentified or incorrectly identified item information for subsequent identification and settlement of the item by the system.
[0042] The present invention proposes an intelligent identification and settlement system for multiple items, which solves the problem of identifying the type and quantity of items and whether there are obscured items when settling multiple items, and can also assist merchants in entering data on newly listed items (unidentified items). It can be seen from the above technical solutions that in order to achieve the above invention objectives, the main technical contents of the present invention are as follows:
[0043] The multi-item intelligent identification and settlement system proposed in the present invention mainly includes five core parts: an identification unit, a settlement unit, a storage unit, an intelligent control unit and a commodity information input unit. The specific architecture is as follows:
[0044] Identification unit: contains one or more image sensors (such as white light cameras, depth cameras, and infrared cameras) to collect image information of goods on the checkout counter. The image is processed through the built-in algorithm model to identify the name, quantity, and stacking status of the goods.
[0045] Settlement unit: includes weighing unit and calculation unit. The weighing unit is used to measure the total weight of the goods on the checkout counter; the calculation unit calculates the total price of the goods based on the recognition results and the data in the storage unit.
[0046] Storage unit: stores relevant information of the product, including product name, image features, weight, price, product code and product packaging image, etc., to provide data support for identification and settlement.
[0047] Intelligent control unit: As the core processing unit of the system, it is responsible for calling the data in the storage unit, comparing and analyzing the results of the identification unit and the settlement unit, and realizing accurate identification and settlement. At the same time, according to user operations and system feedback, it intelligently adjusts the recognition algorithm model to improve recognition accuracy.
[0048] Product information input unit: usually a scanner, used to scan product barcodes or QR codes so that the system can enter information and settle accounts for unidentified products.
[0049] The present invention proposes a method for intelligent identification and settlement of multiple commodities, comprising the following steps:
[0050] Product placement and preliminary identification: Customers place the products to be settled on the checkout counter. The identification unit collects product images through image sensors and performs preliminary identification of product names and quantities.
[0051] Stacking and occlusion processing: If stacking or occlusion of goods is detected, the intelligent control unit will start a special processing process. First, the number of stacked goods is determined through the intersection-and-union recognition technology; second, for seriously obstructed goods, the customer is prompted to rearrange them or use a barcode scanner to identify them.
[0052] Handling of unidentified or incorrectly identified products: For products that cannot be identified or are incorrectly identified, customers are prompted to use the product information input unit to scan the code or enter the information manually.
[0053] Accurate identification and verification: The intelligent control unit compares the identification results with the data in the storage unit to ensure the accuracy and completeness of the identification.
[0054] Weight verification and settlement: The settlement unit measures the total weight of the goods through the weighing unit and verifies it with the recognition result to ensure the accuracy of the settlement. If the weight difference exceeds the threshold, the system will prompt the customer to recheck the goods.
[0055] Display the settlement list and pay: After all the goods are identified correctly and the weight is verified, the system will automatically calculate the total price of the goods and display the settlement list on the display screen. After the user confirms that it is correct, he can make the payment.
[0056] From the above, it can be seen that the system and method provided by the present invention, by integrating technologies such as image recognition, weighing and intelligent control, realizes the rapid and accurate identification and settlement of multiple commodities. The system has high adaptability and flexibility, and can effectively avoid settlement errors caused by the stacking of multiple commodities. At the same time, it is beneficial for merchants to complete the construction of an unidentified commodity database, improves the recognition accuracy and settlement accuracy of multiple commodities, reduces the cost of merchants, improves the settlement efficiency of the retail industry and the level of automation and intelligence in the retail industry, and can be widely used in retail places such as supermarkets and convenience stores.
[0057] By means of the above technical solution, compared with the prior art, the system and method for intelligent identification and settlement of multiple commodities of the present invention have the following beneficial effects:
[0058] (1) Improved recognition accuracy and settlement accuracy for multiple items: By optimizing the recognition algorithm and introducing stacking / occlusion processing technology, the recognition problem in the case of stacked and occluded items is effectively solved, thereby improving recognition accuracy and settlement accuracy.
[0059] (2) Reduced merchant costs: Reduced returns and compensation issues caused by manual identification errors, and reduced merchant operating costs.
[0060] (3) Improved customer experience: Simplified the checkout process, shortened customer waiting time, and improved customer shopping experience.
[0061] (4) Enhanced system adaptability: Supports rapid identification and settlement of newly listed products, meeting merchants' needs for rapid product iteration.
[0062] (5) Promoted the intelligent upgrading of the retail industry: promoted the development of the retail industry towards automation and intelligence, and improved overall operational efficiency and service levels.
[0063] To sum up, the special intelligent identification and settlement system for multiple commodities of the present invention has the above-mentioned advantages and practical values, and there is no similar design publicly published or used in similar products and methods, so it is indeed innovative. It has great improvements in both methods and functions, great progress in technology, and produces easy-to-use and practical effects. It has many improved functions compared with the existing intelligent identification and settlement system for multiple commodities, and is therefore more suitable for practical use.
[0064] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Figure 1 It is a flow chart of a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0066] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method of the intelligent identification and settlement system for multiple commodities proposed by the present invention is described in detail below in combination with the accompanying drawings and preferred embodiments.
[0067] Example 1
[0068] In this embodiment, a method for intelligent identification and settlement of multiple commodities includes the following steps:
[0069] S1: The customer places the goods to be settled on the settlement counter, and the identification unit detects and identifies the quantity and name of the goods;
[0070] S2: Process each product through the recognition unit to determine whether there are overlapping products. If there are overlapping products, go to step S3; if there are no overlapping products, go to step S4;
[0071] S3: Determine whether the overlapping commodities can be identified by the commodity identification unit, and issue a prompt on the display screen for the commodities that are blocked and cannot be identified, reminding the customer to move the blocked commodities to other locations, and return to step S1; issue a prompt on the display screen for the commodities that are not blocked but still cannot be identified, reminding the customer to enter the commodity information through a barcode scanner or other input device;
[0072] S4: Determine the type and quantity of the goods through the recognition unit, and display the specific content on the display screen;
[0073] Furthermore, in step S1, the recognition unit recognizes the commodities on the checkout counter in the checkout counter area by combining an image detection algorithm, and determines the quantity of the overlapping commodities.
[0074] Furthermore, in step S2, the recognition unit identifies the goods through intersection and union ratio, and determines the number of overlapping goods; the front-end prediction captures the input scene in real time, and the item recognition and pricing module is activated when the weight sensor at the checkout counter senses the weight change. First, under the guidance of multiple detection models, the universal item bounding boxes of the input image are obtained respectively; then, based on the calibration results, multiple sets of item bounding boxes are fused to complete the comprehensive evaluation and obtain a series of bounding box lists. The following description of the detection example takes two cameras (placed up / down) as an example. The specific method is to sort out each upper bounding box and lower bounding box in turn, and make judgments to obtain the item list. The specific judgment process needs to be carried out according to different situations:
[0075] If the intersection-and-union ratio of the upper bounding box and a lower bounding box is greater than the threshold T1 (for example, 0.95), and the occlusion attributes of the upper and lower bounding boxes are both 0, then the two bounding boxes are considered to be the upper and lower surfaces of an object, and an object is added, and the two bounding boxes are removed from the upper and lower bounding box lists respectively;
[0076] If the intersection-and-union ratio of the upper bounding box and a lower bounding box is greater than the threshold T1 (for example, 0.95), and one of the upper and lower bounding boxes has an occlusion attribute of 1, the two bounding boxes are considered to be two mutually occluding objects, and two objects are added, and the two bounding boxes are removed from the upper and lower bounding box lists respectively;
[0077] If the intersection-and-union ratio of the upper / lower bounding box and any lower / upper bounding box is less than the threshold T1 (e.g. 0.95), and the occlusion attributes of the upper / lower bounding boxes are both 0, it is determined that the lower / upper bounding box of the object is not detected, so an object is added and a bounding box is removed from the upper / lower bounding box list;
[0078] If the intersection-and-union ratio of the upper / lower bounding box and any lower / upper bounding box is less than the threshold T1 (for example, 0.95), and one of the upper and lower bounding boxes has an occlusion attribute of 1, then the two bounding boxes of the object are considered to be two mutually occluding objects, and the two objects are added, and the two bounding boxes are removed from the upper and lower bounding box lists respectively.
[0079] Furthermore, in step S3, the commodity recognition unit takes an image of the commodity and compares it with the commodity data in the storage unit to determine whether the obscured commodity can be recognized successfully. If it is recognized successfully, the customer is reminded that the obscured commodity can be placed at the current location.
[0080] Furthermore, in step S4, the quantity recognition unit performs statistical calculations based on the data of the quantity recognition unit and the stacking recognition unit to obtain the precise quantity of the goods.
[0081] Furthermore, in step S4, the commodity identification unit photographs the commodities at the checkout counter, checks the commodity data of each commodity against the database in turn, obtains the type of the commodity, and then obtains the barcode information of the commodity, and obtains the specific parameters of the commodity, including quality and price, according to the barcode.
[0082] Furthermore, the recognition unit acquires images through a camera installed on the checkout counter. At least one camera is provided. When multiple cameras are provided, they are placed on the top layer and side of the checkout counter at different angles. The camera types include but are not limited to white light cameras, depth cameras and infrared cameras.
[0083] Furthermore, the recognition units are all trained by the neural network. The background training first obtains multiple sets of data sets under multiple cameras, and independently trains the model for each camera to train a general object detection model and an object classification model. The general bounding box attributes include the center point cx, cy, width w, height h, and whether there is an occlusion. The classification attributes are used to judge the object category of the bounding box.
[0084] Furthermore, it also includes a weighing unit. There is a fixed weight for each single product. The total weight = the number of categories under different categories * the weight of the single product in each category. If the verification matches, the price is calculated. Otherwise, the customer is reminded to move the goods and calculate the price again.
[0085] Example 2
[0086] like Figure 1 As shown, the intelligent identification and settlement method for multiple commodities includes the following steps:
[0087] S1: The customer places the goods to be settled on the settlement counter, and the identification unit determines the rough quantity of the current goods;
[0088] The recognition unit recognizes the commodities on the checkout counter in the effective area of the checkout counter by combining an image detection algorithm, and determines the quantity of the overlapping commodities.
[0089] S2: Process each product through the identification unit to determine whether there are seriously overlapping products. If there are overlapping products, go to step S3; if there are no overlapping products, go to step S4;
[0090] The recognition unit identifies the goods through intersection and union ratio, and determines the number of overlapping goods; the front-end prediction captures the input scene in real time, and when the weight sensor at the checkout counter senses the weight change, the item recognition and pricing module is activated. First, under the guidance of multiple detection models, the universal item bounding boxes of the input image are obtained respectively; then, based on the calibration results, multiple sets of item bounding boxes are fused to complete the comprehensive evaluation and obtain a series of bounding box lists. The following description of the detection example takes two cameras (placed up / down) as an example. The specific method is to sort out each upper bounding box and lower bounding box in turn, and make judgments to obtain the item list. The specific judgment process needs to be carried out according to different situations:
[0091] If the intersection-and-union ratio of the upper bounding box and a lower bounding box is greater than the threshold T1 (for example, 0.95), and the occlusion attributes of the upper and lower bounding boxes are both 0, then the two bounding boxes are considered to be the upper and lower surfaces of an object, and an object is added, and the two bounding boxes are removed from the upper and lower bounding box lists respectively;
[0092] If the intersection-and-union ratio of the upper bounding box and a lower bounding box is greater than the threshold T1 (for example, 0.95), and one of the upper and lower bounding boxes has an occlusion attribute of 1, the two bounding boxes are considered to be two mutually occluding objects, and two objects are added, and the two bounding boxes are removed from the upper and lower bounding box lists respectively;
[0093] If the intersection-and-union ratio of the upper / lower bounding box and any lower / upper bounding box is less than the threshold T1 (e.g. 0.95), and the occlusion attributes of the upper / lower bounding boxes are both 0, it is determined that the lower / upper bounding box of the object is not detected, so an object is added and a bounding box is removed from the upper / lower bounding box list;
[0094] If the intersection-and-union ratio of the upper / lower bounding box and any lower / upper bounding box is less than the threshold T1 (for example, 0.95), and one of the upper and lower bounding boxes has an occlusion attribute of 1, then the two bounding boxes of the object are considered to be two mutually occluding objects, and the two objects are added, and the two bounding boxes are removed from the upper and lower bounding box lists respectively.
[0095] S3: Determine whether the overlapping commodities can be identified by the identification unit, and issue a prompt on the display screen for the commodities that are blocked and cannot be identified, reminding the customer to move the blocked commodities to other locations, and return to step S1; issue a prompt on the display screen for the commodities that are not blocked but still cannot be identified, reminding the customer to scan the commodities separately for settlement through the scanning unit of the settlement unit;
[0096] The commodity recognition unit takes an image of the commodity and compares it with the commodity data stored in the storage unit to determine whether the obscured commodity can be recognized successfully. If it is recognized successfully, the customer is reminded that the obscured commodity can be placed at the current location.
[0097] S4: Determine the type of the commodity through the commodity identification unit, determine the quantity of each commodity through the quantity identification unit, and display the specific content on the display screen;
[0098] The commodity identification unit takes pictures of the commodities at the checkout counter, checks the commodity data of each commodity against the commodity data in the database in turn, obtains the type of the commodity, and then obtains the barcode information of the commodity, and obtains the specific parameters of the commodity according to the barcode, including quality and price.
[0099] The identification unit performs statistical calculations to obtain the precise quantity of the goods.
[0100] It also includes a weight unit. There is a fixed weight for each single product. The total weight = the number of categories under different categories * the weight of the single product in each category. If the verification matches, the price is calculated. Otherwise, the customer is reminded to move the goods and the price is calculated again.
[0101] For commodities settled by the scanning unit, images are collected through the commodity recognition unit and stored in the database.
[0102] In the present invention, the recognition unit collects images through a camera installed on the checkout counter. At least one camera is provided. If multiple cameras are provided, they are placed at different angles of the checkout counter, such as the top floor and the side. The camera types include but are not limited to white light cameras, depth cameras and infrared cameras.
[0103] The recognition units are all trained by the neural network in the background. The background training first obtains multiple sets of data sets under multiple cameras, and independently trains the model for each camera to train a general object detection model. The general bounding box attributes include the center point cx, cy, width w, height h, and whether there is an occlusion. The classification attributes are based on the object category judgment of the bounding box, where object classification includes but is not limited to data detection through the background database.
[0104] Example 3
[0105] This example provides a system and method for intelligent identification and settlement of multiple commodities, which includes the following parts:
[0106] 1. System hardware configuration
[0107] Checkout counter: Made of durable material, with grids on the surface to facilitate the initial counting of quantity identification units. At the same time, built-in weight sensor to support weighing function.
[0108] Camera: Multiple cameras (such as white light camera, depth camera and infrared camera) are installed on the top and sides of the checkout counter to ensure that product image information is captured from multiple angles. The camera is connected to the intelligent control unit via USB or network interface.
[0109] Barcode scanner: Placed near the checkout counter, it is convenient for customers to scan the barcode or QR code of unrecognized products. The barcode scanner is connected to the intelligent control unit via a USB interface.
[0110] Display screen: used to display recognition results, settlement lists and system prompt information. The display screen is connected to the intelligent control unit via an HDMI or VGA interface.
[0111] Intelligent control component: It can call the data and signals of other components. Its built-in algorithm can determine the situation of product stacking, incorrectly input products and unrecognized products, thereby instructing the recognition unit and settlement unit to perform different operations for different situations.
[0112] 2. Software algorithm implementation
[0113] Image recognition algorithm: Developed based on deep learning frameworks (such as TensorFlow and PyTorch), using convolutional neural networks (CNN) to extract features and classify product images. Model training is performed through a large number of annotated data sets to improve recognition accuracy.
[0114] Intersection over Union (IoU) recognition algorithm: For stacked products, the intersection over union (IoU) calculation method is used to determine the overlap of products. The number of products is corrected according to the degree of overlap.
[0115] Weight verification algorithm: Combine the weight information in the product database with the actual weighing result for verification. If the difference exceeds the preset threshold, the customer is prompted to recheck the product.
[0116] 3. System operation process
[0117] The customer places the product at the checkout counter: the system automatically starts the identification process.
[0118] Initial identification and stacking / occlusion processing:
[0119] The identification unit performs preliminary identification of the goods and determines the quantity and name of the goods.
[0120] If stacking or obstruction is detected, the intelligent control unit starts a special processing process to determine the number of stacked products through intersection-over-union recognition technology; prompting customers to re-place the obstructed products;
[0121] If an unrecognized product is detected or the information of the identified product is inconsistent with the information stored in the database, the intelligent control unit starts a program to prompt the customer to scan with a barcode scanner or manually enter the information.
[0122] Accurate identification and verification:
[0123] The intelligent control unit compares and verifies the recognition result with the data in the storage unit to ensure the accuracy of the recognition, and stores it in the commodity database (i.e., the storage unit).
[0124] Weight verification and settlement:
[0125] The settlement unit measures the total weight of the goods through a weighing unit and verifies it with the recognition result.
[0126] If the weight difference exceeds the threshold, the customer is prompted to recheck the product.
[0127] Complete settlement and payment: Complete settlement on the display screen, including information such as product name, unit price, quantity and total price.
[0128] The customer proceeds with the payment after confirmation.
[0129] Update inventory information and generate transaction records.
[0130] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes by using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.
Claims
1. An intelligent identification and settlement system for multiple commodities, the system comprising an identification unit, a settlement unit and a storage unit; the identification unit generates a signal after identifying the multiple commodities and transmits the signal to the settlement unit, and the settlement unit calls the data of the storage unit to complete the identification and settlement of the multiple commodities.
2. The settlement system according to claim 1, wherein the recognition unit comprises one or more image sensors for detecting and recognizing the quantity and name of the goods.
3. The intelligent identification and settlement system according to claim 2, wherein: By designing and arranging the number and positions of the image sensors, accurate identification of multiple items can be achieved.
4. The intelligent identification and settlement system according to claim 3, wherein: The design and arrangement of the number and positions of the image sensors can realize the recognition of the names and quantities of the stacked commodities.
5. The intelligent identification and settlement system according to claim 1, wherein the settlement unit comprises a weighing unit to realize weighing the weight of multiple items that need to be identified.
6. The intelligent identification and settlement system according to claim 1, wherein: The data stored in the storage unit is used for identifying the commodity and settling the commodity price, and the data includes the commodity name, weight, price, code and the outer packaging image of the commodity.
7. The intelligent identification and settlement system according to claim 1, wherein: The system further includes an intelligent control unit, which retrieves the information of the goods in the storage unit according to the recognition result of the recognition unit, and compares the weight with the multiple goods obtained by the settlement unit, thereby realizing accurate recognition and settlement of the multiple goods.
8. The intelligent identification and settlement system according to claim 7, wherein: The system further includes a product information input unit, which is used for the user to scan the coding information of the product that is not recognized or recognized incorrectly by the identification system. The intelligent control unit inputs the information obtained after the user scans into the storage unit to realize the identification of the unrecognized or incorrectly recognized products, the entry and calculation of information, and then the identification system is used for subsequent identification and settlement of the product.
9. The intelligent identification and settlement system according to claim 8, wherein the commodity information input unit is a barcode scanner or a key input device.
10. The intelligent identification and settlement system according to any one of claims 1 to 9, comprising: an identification unit, the identification unit comprising one or more image sensors and a display; A settlement unit, the settlement unit comprising a weighing unit and a calculation unit; A storage unit, wherein the data stored in the storage unit includes the name, weight, price, code and outer packaging image of the commodity; An intelligent control unit, which is capable of calling data from the storage unit, the identification unit, and the settlement unit; A commodity information input unit, wherein the commodity information input unit is used for a user to input information of an unrecognized or incorrectly recognized commodity; in: The data acquired by the identification unit is used for settlement by the settlement unit and comparative analysis by the intelligent control unit, so as to finally determine the name, quantity and price of the commodity; The weighing unit in the settlement unit obtains the weight data of the commodity and transmits it to the intelligent control unit, so that the intelligent control unit compares the quantity and weight of multiple commodities to determine whether the name, quantity and price of the commodity to be settled are correct; and The product information input unit prompts the user to enter the product information based on the information feedback from the intelligent control unit when the product purchased by the user cannot be identified or is identified incorrectly, and stores the information in the storage unit for subsequent identification and settlement of the product by the system.
11. An intelligent identification and settlement system according to any one of claims 1 to 9, wherein the identification unit comprises a quantity identification unit, a stacking identification unit and a commodity identification unit.
12. A method for intelligent identification and settlement of multiple commodities, comprising the following steps: S1: The user places the goods to be settled on the settlement counter for intelligent identification. According to the intelligent identification results, the following steps are performed respectively; S2: If the goods to be settled are identified in step S1 to be severely blocked, so that the goods to be settled cannot be accurately identified, or the quantity and weight of the identified goods are greatly different from the goods information in the storage unit, the user is prompted to rearrange the goods, and then step S1 is repeated to complete the identification of multiple goods; S3: If it is found in step S1 or step S2 that there are unrecognizable or misrecognized commodities, the user is prompted to scan the unrecognized or misrecognized commodities with a barcode scanner, and the unrecognized or misrecognized commodities are corrected, and the system completes data entry at the same time; S4: The intelligent identification unit determines whether all commodities on the checkout counter are correctly identified and the weight verification is passed, and the unit price, quantity and amount of all commodities are calculated.
13. The intelligent identification and settlement method for multiple items according to claim 12, wherein in step S3, the information of the unidentified or incorrectly identified items scanned by the user is stored in the storage unit, thereby realizing the entry and storage of the unidentified or incorrectly identified item information for subsequent identification and settlement of the item by the system.
Citation Information
Patent Citations
Intelligent commodity settlement system and method based on visual recognition and weight sensing technologies
CN108537994A
Self-service settlement method, device and storage medium
CN109214806A
Checkout method, device and system and computer readable storage medium
CN109741119A
Semi-automatic collection system based on commodity image recognition and usage method
CN111028447A
Cashier error correction method and device based on man-machine interaction
CN112668558A
Cited By
Article self-service settlement method and system
CN122290256A
