Shopping mall and supermarket commodity loss prevention method and device
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANSHOW TECH CO LTD
- Filing Date
- 2025-12-24
- Publication Date
- 2026-05-26
Smart Images
Figure CN122090366A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a method and device for preventing damage to supermarket goods. Background Technology
[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.
[0003] Smart shopping carts represent a future trend in smart retail. They automatically recognize shoppers' actions of placing and removing items, as well as identifying products added or removed from the cart. This allows shoppers to directly scan and pay using the smart cart, avoiding queues at checkout during peak hours. While smart shopping carts improve convenience and enhance the shopping experience, loss prevention algorithms have limitations, with upper limits to the accuracy of action and product recognition. For example, shoppers might obstruct the camera when placing items, hide smaller items behind larger ones to prevent the camera from capturing them, or scan a cheaper item and place a similar, more expensive one inside. These behaviors are unavoidable in actual shopping. Therefore, smart shopping carts can only record potentially unusual behaviors to a certain extent, and their loss prevention capabilities need improvement. Summary of the Invention
[0004] This invention provides a method for preventing damage to goods in supermarkets, which can effectively identify suspicious goods in supermarkets and avoid vulnerabilities in the damage prevention of smart shopping carts, including:
[0005] Obtain the first set of product identification results from the shopping cart system for the items in the shopping cart;
[0006] A second set of product identification results is obtained after the intelligent loss prevention device passes through the shopping cart. The first product identification result in the first set of product identification results and the second product identification result in the second set of product identification results both include the product category and the identification confidence level of each product category.
[0007] Based on the product category and recognition confidence, the matching cost for each first product recognition result and each second product recognition result is calculated to form a matching cost matrix;
[0008] Based on the matching cost matrix, the first product identification result and the second product identification result are matched to obtain the matching result, which includes successfully matched products and suspicious products that are not successfully matched.
[0009] This invention provides a supermarket merchandise loss prevention device that effectively identifies suspicious merchandise in supermarkets and avoids vulnerabilities in smart shopping cart loss prevention, including:
[0010] The first product recognition result acquisition module is used to obtain the first product recognition result set of the shopping cart system for the products in the shopping cart;
[0011] The second product identification result acquisition module is used to obtain a second product identification result set of the smart loss prevention device on the products in the shopping cart after the shopping cart passes by. The first product identification result in the first product identification result set and the second product identification result in the second product identification result set both include the product category and the identification confidence level of each product category.
[0012] The matching cost matrix calculation module is used to calculate the matching cost of each first product identification result and each second product identification result based on the product category and the identification confidence, forming a matching cost matrix;
[0013] The matching module is used to match the first product identification result and the second product identification result according to the matching cost matrix to obtain the matching result, which includes successfully matched products and suspicious products that are not successfully matched.
[0014] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for preventing damage to supermarket goods.
[0015] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for preventing damage to supermarket goods.
[0016] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for preventing damage to supermarket goods.
[0017] In this embodiment of the invention, the first product identification result of the shopping cart system is matched with the second product identification result of the intelligent loss prevention device. The advantages of both systems (shopping cart identification is more accurate at close range, while intelligent loss prevention device covers all scenarios) are utilized for cross-validation to reduce the identification bias of a single system (such as missed scans by the shopping cart or misidentification by the intelligent loss prevention device). The matching cost matrix considers both product category (whether they are the same product) and identification confidence (the credibility of the identification result), avoiding the problem of misjudging suspicious items with high identification confidence or missing them with low identification confidence due to category matching alone. This makes the determination of suspicious products more consistent with actual scenarios. Through the matching cost matrix and matching algorithm, automated identification of suspicious products is achieved, reducing the workload of manual verification, especially suitable for supermarkets with high customer traffic. Suspicious products that fail to match are directly marked as suspicious objects, allowing staff to conduct targeted checks, avoiding re-inspection of all products and improving loss prevention response speed. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0019] Figure 1 This is a flowchart of a supermarket merchandise loss prevention method according to an embodiment of the present invention;
[0020] Figure 2 This is a flowchart illustrating the matching process in an embodiment of the present invention;
[0021] Figure 3 This is a schematic diagram of the matching results in an embodiment of the present invention;
[0022] Figure 4 This is a schematic diagram of the structure of the supermarket merchandise damage prevention device in an embodiment of the present invention;
[0023] Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0025] Besides smart shopping carts, smart loss prevention devices are also gradually appearing in the public eye. By installing cameras on the top and sides of these devices, when a shopper pushes their shopping cart past, the cameras automatically capture images of the items in the cart and perform corresponding product identification. If the identification results from the smart loss prevention device differ from those on the shopper's shopping list, the system will automatically alert inspectors to conduct a random check of the items in the shopper's cart.
[0026] While both smart shopping carts and smart loss prevention devices can analyze shopper behavior and purchased goods to some extent and provide timely feedback to inspectors, each method has its limitations. For example, if a shopper obstructs the camera while shopping, as mentioned earlier, the loss prevention algorithm of the smart shopping cart becomes completely ineffective. Smart loss prevention devices can only identify and analyze goods on the surface of the shopping cart, and cannot identify goods hidden inside the cart.
[0027] To mitigate the limitations of different smart products' capabilities, this invention proposes a loss prevention solution combining smart loss prevention equipment and a smart shopping cart. Based on the above analysis, while a smart shopping cart can analyze a shopper's entire shopping process, identifying and analyzing their shopping behavior and purchased goods, it cannot prevent the aforementioned abnormal shopping behaviors. A smart loss prevention device can identify and analyze goods on the surface of the shopping cart, but it cannot identify goods inside the cart. Therefore, integrating the shopping results from the smart shopping cart with the analysis results from the smart loss prevention device can overcome the shortcomings of the two aforementioned solutions.
[0028] In this embodiment of the invention, the intelligent shopping cart is a smart shopping cart capable of automatic barcode scanning and checkout, and of recognizing the shopper's behavior and the products during the shopping process. Behavior recognition refers to recognizing the shopper's actions of placing items into or removing them from the shopping cart. Product recognition refers to recognizing the products placed into or removed from the shopping cart by the shopper. The intelligent loss prevention device can be an intelligent detection gate installed at the entrance of a supermarket that automatically captures images of the products in the shopping cart and identifies them, or it can be other types of equipment.
[0029] Figure 1 This is a flowchart of a supermarket merchandise loss prevention method according to an embodiment of the present invention. The method is applied to a backend server and includes:
[0030] Step 101: Obtain the first set of product identification results from the shopping cart system for the products in the shopping cart;
[0031] Step 102: Obtain a second set of product identification results for the products in the shopping cart after the smart loss prevention device passes through the shopping cart. Both the first and second product identification results include the product category and the identification confidence level for each product category.
[0032] Step 103: Based on the product category and recognition confidence, calculate the matching cost for each first product recognition result and each second product recognition result to form a matching cost matrix;
[0033] Step 104: Match the first product identification result and the second product identification result according to the matching cost matrix to obtain the matching result, which includes successfully matched products and suspicious products that were not successfully matched.
[0034] In this embodiment of the invention, the first product identification result of the shopping cart system is matched with the second product identification result of the intelligent loss prevention device. The advantages of both systems (shopping cart identification is more accurate at close range, while intelligent loss prevention device covers all scenarios) are utilized for cross-validation to reduce the identification bias of a single system (such as missed scans by the shopping cart or misidentification by the intelligent loss prevention device). The matching cost matrix considers both product category (whether they are the same product) and identification confidence (the credibility of the identification result), avoiding the problem of misjudging suspicious items with high identification confidence or missing them with low identification confidence due to category matching alone. This makes the determination of suspicious products more consistent with real-world scenarios. Through the matching cost matrix and matching algorithm, automated identification of suspicious products is achieved, reducing the workload of manual verification, especially suitable for high-traffic supermarkets. Suspicious products that fail to match are directly marked as suspicious objects, allowing staff to conduct targeted checks, avoiding re-inspection of all products and improving loss prevention response speed.
[0035] In step 101, the first set of product identification results of the shopping cart system for the products in the shopping cart is obtained;
[0036] The shopping cart system includes a shopping cart, a camera, and a first recognition device. As the shopping cart is pushed, the camera on the cart captures the shopper's actions. The first recognition device then identifies the captured data to obtain a first product recognition result. This first product recognition result includes the product category obtained from the recognized product image and the recognition confidence level. For example, if three product categories are identified, it is represented as follows: The corresponding recognition confidence level can be expressed as The confidence level is greater than or equal to 0 and less than or equal to 1.
[0037] In one embodiment, obtaining a first set of product identification results for the items in the shopping cart by the shopping cart system includes:
[0038] The system obtains the product scanning information and corresponding first timestamp sent by the shopping cart system, as well as the shopper's shopping behavior data and corresponding second timestamp. The scanning information includes the product identification result, and the shopping behavior data includes the shopper's behavior identification result of putting the product into the shopping cart and the product identification result.
[0039] Based on the scanned information, a first product identification result is generated. The product category in the first product identification result is the same as the product category in the product identification result of the scanned information, and the identification confidence of this product category is set to 1.
[0040] If, based on the first timestamp, there is no QR code scanning information within a first preset time period before the second timestamp corresponding to the shopping behavior data, then the product identification result in the shopping behavior data will be used as the first product identification result.
[0041] In this embodiment of the invention, the shopping cart system further includes a scanning dock, whereby shoppers use the scanning dock to scan the purchased goods to obtain scanning information and record a first timestamp; at the same time, a camera on the shopping cart captures the shopper's shopping behavior, and a first recognition device of the shopping cart system recognizes the captured shopping behavior to obtain shopping behavior data and record a second timestamp; the shopping cart system sends the scanning information and the corresponding first timestamp, as well as the shopping behavior data and the corresponding second timestamp, to the backend server. The system can determine whether a shopper added an item to their shopping cart within a first preset time period (e.g., 5 seconds) after scanning a product's barcode, based on the first and second timestamps. There are two scenarios: First, if the camera captured the shopper adding the item to their cart within the first preset time period (e.g., 5 seconds), the product category obtained from the barcode scan is used as the product category, and the recognition confidence level is set to 1, thus obtaining the first product recognition result. Second, if the camera did not capture the shopper adding an item to their cart within the first preset time period (e.g., 5 seconds), the product category obtained from the barcode scan is still used as the product category, and the recognition confidence level is set to 1, thus obtaining the first product recognition result. If there is no barcode scanning information within the first preset time period before the second timestamp of the shopper's shopping behavior data, the first product recognition result from the shopping behavior data obtained by the first recognition device is used as the first product recognition result for that product. Any product category identified by the dock scanning tool will be directly added to the shopping list. If the product is identified by the camera (e.g., there are 3 product categories), and there is no scan information within the first preset time period before the second timestamp of the shopper's shopping behavior data is captured, then the product with the highest confidence level will be added to the shopping list.
[0042] The shopping behavior data also includes the recognition results of the shopper removing items from the shopping cart and the product recognition results. In the case of removal, without the participation of the camera, the product category with the highest confidence in the product category recognized by the camera is directly compared with the product category of the items in the shopping list, and the corresponding items in the shopping list are deleted.
[0043] In one embodiment, the method further includes:
[0044] If shopping behavior data exists within the first preset time period after the first timestamp of the scanned information, determine whether the product category corresponding to the scanned information and the product category with the highest recognition confidence corresponding to the shopping behavior data are consistent.
[0045] If not, mark the scanned information as abnormal shopping behavior.
[0046] The above information on abnormal shopping behavior can be used to report to verification personnel later.
[0047] If shopping behavior data exists within the first preset time period (e.g., within 5 seconds) after the first timestamp corresponding to the scanned information, it is generally assumed that the product corresponding to the scan action and the product captured by the camera are the same product (the shopper scans the product first and then adds it to the shopping cart). In this case, since the scanned information is more accurate, the product category identified by the scanner is directly added to the shopping list. However, the possibility of abnormal scanned information cannot be ruled out. Therefore, when the product category identified by the scanner is inconsistent with the product category with the highest confidence level identified by the camera within the first preset time period after the scan, in order to further improve the accuracy of loss prevention, the scanned information can be marked as abnormal shopping behavior information. The above-mentioned abnormal shopping behavior information can be used for subsequent reporting to verification personnel.
[0048] In step 102, a second set of product identification results is obtained after the smart loss prevention device passes through the shopping cart. Both the first and second product identification results include the product category and the identification confidence level of each product category.
[0049] In this embodiment of the invention, the intelligent loss prevention device can be a loss prevention door or other device capable of scanning items in a shopping cart.
[0050] In one embodiment, obtaining a second set of product identification results from the smart loss prevention device after the shopping cart has passed by includes:
[0051] Obtain a first set sent by the intelligent loss prevention device, wherein the first set is a set of product identification results formed by reading products with RFID tags through an RFID reader;
[0052] Obtain a second set sent by the intelligent loss prevention device, which is a set of product recognition results generated by taking pictures of the products in the shopping cart through a camera;
[0053] If the product with the highest recognition confidence in a product recognition result in the first set also exists in a product recognition result in the second set, delete that product recognition result in the second set.
[0054] Merge all product identification results from the first and second sets to obtain the second product identification result for the products in the shopping cart.
[0055] In the above embodiments, the existing RFID tags on the goods (which are widely used in many supermarkets for loss prevention) are utilized to integrate a high-performance RFID reader (multi-antenna array) into the smart loss prevention equipment. When the shopping cart passes by, the reader can penetrate the product packaging and obstructions (such as plastic bags and other goods) to read the product IDs of all products with RFID tags, thereby obtaining the product category and setting the identification confidence level of the product category to 1.
[0056] Smart loss prevention equipment uses cameras from various angles to photograph and analyze the items in the shopping cart as it passes by. Specifically, the top camera captures the items on the top layer of the cart, while the side cameras capture visible items on the sides. The product identification and analysis typically involves first detecting the products in the captured images and labeling each item. Then, each product is identified, generally in two ways: one is direct product identification, such as using feature retrieval; the other is capturing the barcode (including barcodes, QR codes, etc.) on the product image and then identifying the barcode. Generally, direct product identification outputs multiple product categories and corresponding recognition confidence scores; barcode identification outputs a single product identification result, directly obtaining the product category, with a corresponding recognition confidence score of 1.
[0057] Because RFID readers and cameras can overlap in their identification processes, and camera identification involves image recognition with lower confidence than RFID readers, the product identification result from the RFID reader should be retained when there is overlap.
[0058] Since smart shopping carts are used inside supermarkets—that is, shoppers push the carts while shopping—while smart loss prevention devices are used at supermarket entrances and exits, the two smart algorithms can only be executed separately. This results in the smart shopping cart generating M product recognition results during the entire shopping process, while the smart loss prevention device generates N product recognition results as the cart passes by. The number and content of products recognized by the two algorithms may be different. Therefore, it is necessary to match these M product recognition results with the N product recognition results as much as possible to find the commonly recognized products and the suspicious products.
[0059] The process of obtaining the second product identification result described above uses both RFID tags and cameras. In another embodiment, the identification results of all products in the first set can be used as the second product identification result of the products in the shopping cart. In yet another embodiment, the identification results of all products in the second set can be used as the second product identification result of the products in the shopping cart.
[0060] In step 103, based on the product category and recognition confidence, the matching cost of each first product recognition result and each second product recognition result is calculated to form a matching cost matrix;
[0061] In one embodiment, the method further includes:
[0062] After obtaining the first set of product identification results of the shopping cart system for the products in the shopping cart, if the number of product categories in the first product identification result of a product is less than the preset number, it is supplemented with a value of 0. The product categories in the first product identification result of the products in the shopping cart are sorted from high to low according to the identification confidence.
[0063] After obtaining the product recognition results of the smart loss prevention device after it passes through the shopping cart, if the number of product categories in the second product recognition result of a product is less than the preset number, it is supplemented with a value of 0. The product categories in the second product recognition result are sorted from high to low according to the recognition confidence.
[0064] In the above embodiments, if the preset number of product categories in the first product identification result is 3, then it is necessary to retain three product categories in the first product identification result. In the case of a scanning dock, if the number of product categories in the first product identification result for a single product is only 1, and the identification confidence level is 1, then it is necessary to retain three product categories in the product identification result (represented as...). ) and the corresponding recognition confidence (expressed as ),at this time, Fill in 0 in the corresponding field, and the corresponding It is also 0. The second recognition result is processed similarly.
[0065] In one embodiment, the matching cost for each first product identification result and each second product identification result is calculated using the following formula, based on the product category and identification confidence level:
[0066]
[0067] in, The matching cost is the cost of the first product identification result C and the second product identification result D. The degree of matching between the first product identification result C and the second product identification result D. and Here, n is the coefficient, and n is the preset quantity; For the i-th product category in the first product identification result C, For the j-th product category in the second product identification result D, for The corresponding recognition confidence level, for The corresponding recognition confidence level;
[0068] Taking a preset quantity of 3 as an example, the recognition results for a certain product may be 1, 2, or 3. The top 3 recognition results are sorted from highest to lowest confidence level. Therefore, the first recognition has the highest confidence level, and the third recognition has the lowest confidence level. Therefore, a set of parameters is set... As the coefficient at each position, it satisfies the following condition. . express The corresponding coefficients, express The corresponding coefficients. If i=1, j=3, that is, calculate... and Assuming , For the first item recognition result in the smart shopping cart A second item identification result in the intelligent loss prevention equipment The confidence levels for identification are respectively and Calculate matching degree .because ,and ,so .
[0069] Since the recognition models of shopping carts and smart loss prevention devices may have systematic biases (such as the shopping cart model giving higher scores to items that are close at hand), two recognition confidence mapping functions can be constructed through maximum likelihood estimation to transform the recognition confidence of both parties into a unified probability space before calculating the matching degree.
[0070] In one embodiment, the method further includes:
[0071] Before calculating the matching cost of each first product identification result and each second product identification result, the mapping value of each identification confidence in each first product identification result is calculated through the shopping cart identification confidence mapping function, and the mapping value of each identification confidence in each second product identification result is calculated through the intelligent loss prevention device identification confidence mapping function.
[0072] Based on the product category and recognition confidence level, the matching cost for each first product recognition result and each second product recognition result is calculated, including:
[0073] Based on the mapping value between product category and recognition confidence, calculate the matching cost for each first product recognition result and each second product recognition result.
[0074] In this embodiment of the invention, the process of constructing the shopping cart identification confidence mapping function and the intelligent loss prevention device identification confidence mapping function through maximum likelihood estimation is as follows:
[0075] The core logic of constructing a confidence mapping function for shopping cart recognition and a confidence mapping function for smart loss prevention equipment using maximum likelihood estimation to transform the recognition confidence of shopping carts and smart loss prevention equipment to a unified probability space is based on the assumption that the recognition confidence of the same product in different recognition systems (shopping cart / smart loss prevention equipment) should correspond to the same true probability. The parameters of the mapping function are estimated using historical data to make the mapped value of each recognition confidence more closely approximate the true probability distribution. The recognition models of shopping carts and smart loss prevention equipment have systematic biases (e.g., shopping carts tend to score nearby products higher), leading to different true probabilities p for x and y (e.g., x=0.8 in the shopping cart may correspond to a true probability p=0.7, while y=0.8 in the smart loss prevention equipment may correspond to p=0.9). This bias needs to be eliminated through the mapping function. The detailed logical steps are as follows:
[0076] (1) Collect training data
[0077] Historical data on the same product being simultaneously identified by both the shopping cart and the smart loss prevention device in the same scenario needs to be collected. The data format is as follows: ,in:
[0078] : Confidence level of the shopping cart in recognizing the i-th item ( ∈[0,1]);
[0079] : The confidence level of the intelligent loss prevention device in identifying the i-th item ( ∈[0,1]);
[0080] The probability (label value) that the product is "correctly identified".
[0081] If the product is identified via barcode scanning (such as the first item in the shopping cart), then =1 (Barcode recognition is absolutely reliable);
[0082] If the product has no barcode (such as the second product identification result of a smart loss prevention device), it can be manually verified and labeled (if correctly identified). =1, if incorrect =0);
[0083] For ambiguous scenarios (such as partial correct recognition), the average accuracy of multiple recognitions can be used to approximate the result. .
[0084] (2) Assume the form of the mapping function
[0085] The mapping function must satisfy monotonicity (the higher the recognition confidence, the higher the true probability after mapping) and range constraints (the mapping result ∈ [0,1]). In this embodiment of the invention, a linear function is used as the mapping function, and is defined as:
[0086] (a,c>0 ensures monotonicity, and the range is constrained by parameters a,b,c,d).
[0087] (3) Define the probability model (the distribution assumption of the mapping error)
[0088] Assume the error of the mapping function follows a normal distribution (since the identification confidence bias is usually random noise):
[0089] Shopping cart recognition confidence Compared with the true probability Relationship: yes Add noise, i.e. ,in ( It is the inverse function of f, representing the confidence level of shopping cart recognition corresponding to the true probability p).
[0090] Confidence level of intelligent loss prevention equipment Compared with the true probability Relationship: ,in ( (This refers to independent and identically distributed noise).
[0091] like Then the inverse function ,therefore inverse function , .
[0092] (4) Construct the likelihood function
[0093] The likelihood function describes the probability of historical data occurring given the parameters of a mapping function. The goal is to find the parameters that maximize the likelihood function (i.e., the parameters most likely to generate the observed data).
[0094] For the above linear mapping function, the parameters are: ;
[0095] The i-th sample The joint probability density is:
[0096]
[0097] in, It is a normal distribution The probability density function, It is a normal distribution The probability density function.
[0098] The overall likelihood function, which is the product of the joint probabilities of all samples, is as follows:
[0099]
[0100] (5) Maximize the likelihood function (estimate parameters)
[0101] To simplify calculations, the logarithm of the likelihood function is usually taken (the log-likelihood function has the same monotonicity as the original function), transforming the product into a summation:
[0102]
[0103] The parameters are solved by maximizing the log-likelihood function using numerical optimization methods (such as gradient descent and Newton's method). This yields the final mapping function:
[0104] Shopping cart recognition confidence mapping: ;
[0105] Intelligent loss prevention equipment identification confidence mapping: .
[0106] (6) Verify the validity of the mapping function
[0107] Validation logic: Using test data not used in training, check whether the mapped recognition confidence scores f(x) and g(y) are closer to the true probability p (e.g., through mean squared error). measure).
[0108] If the deviation is large, the form of the mapping function needs to be readjusted (e.g., by using the Sigmoid function) or additional training data needs to be provided.
[0109] After obtaining the shopping cart recognition confidence mapping function and the smart loss prevention device recognition confidence mapping function, the recognition confidence x in the first item recognition result of the shopping cart is converted into a probability in a unified space: The recognition confidence y in the second product recognition result of the intelligent loss prevention device is converted into a probability in a unified space: ; after conversion and It can be used directly to calculate the matching degree. .
[0110] In step 104, the first product identification result and the second product identification result are matched according to the matching cost matrix to obtain the matching result, which includes successfully matched products and suspicious products that are not successfully matched.
[0111] Figure 2 This is a flowchart illustrating the matching process in one embodiment of the present invention. In one embodiment, the first product identification result and the second product identification result are matched according to the matching cost matrix to obtain a matching result. The matching result includes successfully matched products and unmatched suspicious products, including:
[0112] Step 201: Adjust the matching cost matrix according to the predefined associated products;
[0113] For the matching cost in the matching cost matrix, check whether the products in the first product identification result and the products in the second product identification result involved in the matching cost are predefined related products. If so, adjust the matching cost. The adjustment formula is as follows:
[0114]
[0115] in, Let β be the co-occurrence probability of items k and j, and let β be the weight (e.g., 0.5).
[0116] Related products refer to goods that are frequently selected together during a user's shopping process and have a correlation in consumption behavior. For example, bread and milk, as mentioned in the document, are often purchased together by consumers as a breakfast combo and are typical related products. Similarly, shampoo and conditioner, toothbrushes and toothpaste, potato chips and carbonated drinks, etc., will show a high co-occurrence probability (i.e., the probability of them appearing in the same order or the same shopping process) in shopping data because their usage scenarios are complementary and their consumption habits are related.
[0117] Step 202: From the first product identification results, select the first product identification results with the highest identification confidence greater than a preset threshold, and denot them as the third set. From the second product identification results, select the second product identification results with the highest identification confidence greater than a preset threshold, and denot them as the fourth set. Each first product identification result in the third set corresponds to a row in the matching cost matrix, and each second product identification result in the fourth set corresponds to a column. The first product identification result and the second product identification result at the intersection of the row and the column are the target matching pair.
[0118] Step 203: Construct the first submatrix from all target matching pairs, and use the task allocation matching algorithm to determine the accuracy of the target matching pairs. If the accuracy is accurate, determine the successfully matched product corresponding to the target matching pair; otherwise, determine the suspicious product that was not successfully matched corresponding to the target matching pair.
[0119] In this embodiment of the invention, the task allocation matching algorithm is preferably the Hungarian algorithm, but it can also be the Kuhn-Munkres algorithm, Hopcroft-Karp algorithm, Blossom Algorithm, lossom-W Algorithm, etc., without limitation. Besides the above method for matching the first and second product identification results, in another embodiment, the associated products can be disregarded, and the task allocation matching algorithm of steps 202 and 203 can be used directly based on the matching cost matrix to match the first and second product identification results. In yet another embodiment, steps 201-203 can be omitted, and the task allocation matching algorithm can be used directly based on the matching cost matrix to match the first and second product identification results.
[0120] Through the above embodiments, successfully matched products and unmatched suspicious products are obtained. Successfully matched products indicate that the product identified by the smart shopping cart and the product identified by the smart loss prevention device are the same product. Unmatched suspicious products are products that may have problems and need to be reported.
[0121] In one embodiment, the method further includes:
[0122] Obtain information on abnormal shopping behavior identified by the shopping cart system;
[0123] Get the shopping list sent by the shopping cart system;
[0124] Analyze the correspondence between the matching results and the shopping list to obtain information on abnormal shopping lists;
[0125] Send the abnormality list information and abnormal shopping behavior information to the verification personnel.
[0126] In the above embodiments, the abnormal shopping behavior information sent to the verification personnel refers to the fact that during the process of obtaining the first product identification result, the camera captured the product (shopping behavior data) within a first preset time period (e.g., within 10 seconds) after the first timestamp corresponding to the scanning information, but the product category corresponding to the scanning information and the product category with the highest confidence in the product category corresponding to the shopping behavior data are inconsistent. At this time, the scanning information is marked as abnormal shopping behavior information.
[0127] The items in the shopping list are derived from the first set of product recognition results of the shopping cart system. There are two cases: First, any product category recognized by the barcode scanner is directly added to the shopping list; Second, if the product is recognized by the camera (for example, there are 3 product categories), and there is no scan information within the first preset time period before the second timestamp of the shopper's shopping behavior data, then the product category with the highest confidence among the 3 product categories is selected and added to the shopping list.
[0128] The abnormal list information includes three situations: First, the quantity of the successfully matched products is inconsistent with the corresponding products in the shopping list; second, the successfully matched products do not exist in the shopping list; and third, suspicious products that failed to match.
[0129] Figure 3 This diagram illustrates the matching results in an embodiment of the present invention. A, B, K, H, E, and G are all products. The first product identification result is matched with the second product identification result to obtain the matching result. Products A, B, K, and H are successfully matched. The matching result shows that there are 2 products B, but only 1 product B is listed in the shopping list. This mismatch between the quantity of product B and the shopping list constitutes an abnormal list information and needs to be reported to the verification personnel. Product H is a successfully matched product, but it is not listed in the shopping list, also constituting an abnormal list information and requiring reporting to the verification personnel. Products E and G are suspicious products that failed to match, also constituting abnormal list information and requiring reporting to the verification personnel.
[0130] This invention also proposes a supermarket merchandise damage prevention device, the principle of which is similar to the supermarket merchandise damage prevention method, and will not be described in detail here.
[0131] Figure 4 This is a schematic diagram of the structure of the supermarket merchandise damage prevention device in an embodiment of the present invention, including:
[0132] The first product recognition result acquisition module 401 is used to obtain the first product recognition result set of the shopping cart system for the products in the shopping cart;
[0133] The second product identification result acquisition module 402 is used to obtain a second product identification result set of the smart loss prevention device on the products in the shopping cart after the shopping cart passes by. The first product identification result in the first product identification result set and the second product identification result in the second product identification result set both include the product category and the identification confidence level of each product category.
[0134] The matching cost matrix calculation module 403 is used to calculate the matching cost of each first product identification result and each second product identification result based on the product category and the identification confidence, and form a matching cost matrix.
[0135] The matching module 404 is used to match the first product identification result and the second product identification result according to the matching cost matrix to obtain the matching result, which includes successfully matched products and unmatched suspicious products.
[0136] In one embodiment, the first product identification result acquisition module is used to:
[0137] The system obtains the product scanning information and corresponding first timestamp sent by the shopping cart system, as well as the shopper's shopping behavior data and corresponding second timestamp. The scanning information includes the product identification result, and the shopping behavior data includes the shopper's behavior identification result of putting the product into the shopping cart and the product identification result.
[0138] Based on the scanned information, a first product identification result is generated. The product category in the first product identification result is the same as the product category in the product identification result of the scanned information, and the identification confidence of this product category is set to 1.
[0139] If, based on the first timestamp, there is no QR code scanning information within a first preset time period before the second timestamp corresponding to the shopping behavior data, then the product identification result in the shopping behavior data will be used as the first product identification result.
[0140] In one embodiment, the first product identification result acquisition module is used to:
[0141] If shopping behavior data exists within the first preset time period after the first timestamp of the scanned information, determine whether the product category corresponding to the scanned information and the product category with the highest recognition confidence corresponding to the shopping behavior data are consistent.
[0142] If not, mark the scanned information as abnormal shopping behavior.
[0143] In one embodiment, the second product identification result acquisition module is used to:
[0144] Obtain a first set sent by the intelligent loss prevention device, wherein the first set is a set of product identification results formed by reading products with RFID tags through an RFID reader;
[0145] Obtain a second set sent by the intelligent loss prevention device, which is a set of product recognition results generated by taking pictures of the products in the shopping cart through a camera;
[0146] If the product with the highest recognition confidence in a product recognition result in the first set also exists in a product recognition result in the second set, delete that product recognition result in the second set.
[0147] Merge all product identification results from the first and second sets to obtain the second product identification result for the products in the shopping cart.
[0148] In one embodiment, the matching cost matrix calculation module is further configured to:
[0149] After obtaining the first set of product identification results of the shopping cart system for the products in the shopping cart, if the number of product categories in the first product identification result of a product is less than the preset number, it is supplemented with a value of 0. The product categories in the first product identification result of the products in the shopping cart are sorted from high to low according to the identification confidence.
[0150] After obtaining the second product identification result set of the smart loss prevention device after passing through the shopping cart, if the number of product categories in the second product identification result of a product is less than the preset number, it is supplemented with a value of 0. The product categories in the second product identification result of the products in the shopping cart are sorted from high to low according to the identification confidence.
[0151] In one embodiment, the matching cost matrix calculation module is further configured to:
[0152] The following formula is used to calculate the matching cost between each first product identification result and each second product identification result, based on the product category and identification confidence:
[0153]
[0154] in, The matching cost is the cost of the first product identification result C and the second product identification result D. The degree of matching between the first product identification result C and the second product identification result D. and Here, n is the coefficient, and n is the preset quantity; For the i-th product category in the first product identification result C, For the j-th product category in the second product identification result D, for The corresponding recognition confidence level, for The corresponding recognition confidence level.
[0155] In one embodiment, the matching cost matrix calculation module is further configured to:
[0156] Before calculating the matching cost of each first product identification result and each second product identification result, the mapping value of each identification confidence in each first product identification result is calculated through the shopping cart identification confidence mapping function, and the mapping value of each identification confidence in each second product identification result is calculated through the intelligent loss prevention device identification confidence mapping function.
[0157] Based on the mapping value between product category and recognition confidence, calculate the matching cost for each first product recognition result and each second product recognition result.
[0158] In one embodiment, the matching module is used to:
[0159] The matching cost matrix is adjusted based on predefined associated products;
[0160] From the first product identification results, the first product identification results with the highest identification confidence greater than a preset threshold are selected and denoted as the third set. From the second product identification results, the second product identification results with the highest identification confidence greater than a preset threshold are selected and denoted as the fourth set. Each first product identification result in the third set corresponds to a row in the matching cost matrix, and each second product identification result in the fourth set corresponds to a column. The first product identification result and the second product identification result at the intersection of the row and the column are the target matching pair.
[0161] The target matching pairs are used to form the first sub-matrix, and the task allocation matching algorithm is used to determine the accuracy of the target matching pairs. If the accuracy is accurate, the target matching pairs are identified as successfully matched products; otherwise, the target matching pairs are identified as unmatched suspicious products.
[0162] The elements of the matching cost matrix other than all target matching pairs are used to form a second submatrix, and the task allocation matching algorithm is used to obtain the successfully matched products and the suspicious products that were not successfully matched.
[0163] In one embodiment, the apparatus further includes a transmitting module for:
[0164] Obtain information on abnormal shopping behavior identified by the shopping cart system;
[0165] Get the shopping list sent by the shopping cart system;
[0166] Analyze the correspondence between the matching results and the shopping list to obtain information on abnormal shopping lists;
[0167] Send the abnormality list information and abnormal shopping behavior information to the verification personnel.
[0168] In summary, the method and apparatus proposed in this invention match the first product identification result of the shopping cart system with the second product identification result of the intelligent loss prevention device. This leverages the advantages of both systems (shopping cart identification is more accurate at close range, while intelligent loss prevention devices cover all scenarios) for cross-validation, reducing the identification bias of a single system (such as missed scans by the shopping cart or misidentification by the intelligent loss prevention device). The matching cost matrix considers both product category (whether they are the same product) and identification confidence (the credibility of the identification result), avoiding misjudgments of high identification confidence as suspicious or missed judgments of low identification confidence due to category matching alone. This makes the determination of suspicious products more consistent with real-world scenarios. Through the matching cost matrix and matching algorithm, automated identification of suspicious products is achieved, reducing the workload of manual verification, especially suitable for high-traffic supermarkets. Suspicious products that fail to match are directly marked as suspicious objects, allowing staff to conduct targeted checks, avoiding re-inspection of all products and improving loss prevention response speed.
[0169] This invention also provides a computer device. Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, it implements the above-mentioned method for preventing damage to supermarket goods.
[0170] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for preventing damage to supermarket goods.
[0171] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for preventing damage to supermarket goods.
[0172] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0173] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0175] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0176] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for preventing damage to supermarket goods, characterized in that, include: Obtain the first set of product identification results from the shopping cart system for the items in the shopping cart; A second set of product identification results is obtained after the smart loss prevention device passes through the shopping cart. The first product identification result in the first product identification result set and the second product identification result in the second product identification result set both include the product category and the identification confidence level of each product category. Based on the product category and recognition confidence, the matching cost for each first product recognition result and each second product recognition result is calculated to form a matching cost matrix; Based on the matching cost matrix, the first product identification result and the second product identification result are matched to obtain the matching result, which includes successfully matched products and suspicious products that are not successfully matched.
2. The method as described in claim 1, characterized in that, Obtain the first set of product identification results from the shopping cart system for the items in the shopping cart, including: The system obtains the product scanning information and corresponding first timestamp sent by the shopping cart system, as well as the shopper's shopping behavior data and corresponding second timestamp. The scanning information includes the product identification result, and the shopping behavior data includes the shopper's behavior identification result of putting the product into the shopping cart and the product identification result. Based on the scanned information, a first product identification result is generated. The product category in the first product identification result is the same as the product category in the product identification result of the scanned information, and the identification confidence of this product category is set to 1. If, based on the first timestamp, there is no QR code scanning information within a first preset time period before the second timestamp corresponding to the shopping behavior data, then the product identification result in the shopping behavior data will be used as the first product identification result.
3. The method as described in claim 2, characterized in that, Also includes: If shopping behavior data exists within the first preset time period after the first timestamp of the scanned information, determine whether the product category corresponding to the scanned information and the product category with the highest recognition confidence corresponding to the shopping behavior data are consistent. If not, mark the scanned information as abnormal shopping behavior.
4. The method as described in claim 1, characterized in that, Obtain a second set of product recognition results from the smart loss prevention device after it passes through the shopping cart, including: Obtain a first set sent by the intelligent loss prevention device, wherein the first set is a set of product identification results formed by reading products with RFID tags through an RFID reader; Obtain a second set sent by the intelligent loss prevention device, which is a set of product recognition results generated by taking pictures of the products in the shopping cart through a camera; If the product with the highest recognition confidence score of a product recognition result in the first set also exists in a product recognition result in the second set, delete that product recognition result in the second set. Merge all product identification results from the first and second sets to obtain the second product identification result for the products in the shopping cart.
5. The method as described in claim 1, characterized in that, Also includes: After obtaining the first set of product identification results of the shopping cart system for the products in the shopping cart, if the number of product categories in the first product identification result of a product is less than the preset number, it is supplemented with a value of 0. The product categories in the first product identification result of the products in the shopping cart are sorted from high to low according to the identification confidence. After obtaining the second product identification result set of the smart loss prevention device after passing through the shopping cart, if the number of product categories in the second product identification result of a product is less than the preset number, it is supplemented with a value of 0. The product categories in the second product identification result of the products in the shopping cart are sorted from high to low according to the identification confidence.
6. The method as described in claim 1, characterized in that, The following formula is used to calculate the matching cost between each first product identification result and each second product identification result, based on the product category and identification confidence: in, The matching cost is the cost of the first product identification result C and the second product identification result D. The degree of matching between the first product identification result C and the second product identification result D. and Here, n is the coefficient, and n is the preset quantity; For the i-th product category in the first product identification result C, For the j-th product category in the second product identification result D, for The corresponding recognition confidence level, for The corresponding recognition confidence level.
7. The method as described in claim 1, characterized in that, Also includes: Before calculating the matching cost of each first product identification result and each second product identification result, the mapping value of each identification confidence in each first product identification result is calculated through the shopping cart identification confidence mapping function, and the mapping value of each identification confidence in each second product identification result is calculated through the intelligent loss prevention device identification confidence mapping function. Based on the product category and recognition confidence level, the matching cost for each first product recognition result and each second product recognition result is calculated, including: Based on the mapping value between product category and recognition confidence, calculate the matching cost for each first product recognition result and each second product recognition result.
8. The method as described in claim 1, characterized in that, Based on the matching cost matrix, the first product identification result and the second product identification result are matched to obtain a matching result. The matching result includes successfully matched products and suspicious products that are not successfully matched, including: The matching cost matrix is adjusted based on predefined associated products; From the first product identification results, the first product identification results with the highest identification confidence greater than a preset threshold are selected and denoted as the third set. From the second product identification results, the second product identification results with the highest identification confidence greater than a preset threshold are selected and denoted as the fourth set. Each first product identification result in the third set corresponds to a row in the matching cost matrix, and each second product identification result in the fourth set corresponds to a column. The first product identification result and the second product identification result at the intersection of the row and the column are the target matching pair. All target matching pairs are used to form the first submatrix. The task allocation matching algorithm is then used to determine the accuracy of the target matching pairs. If the matching pairs are accurate, the products corresponding to the target matching pairs are determined to be successfully matched. Otherwise, the products corresponding to the target matching pairs are determined to be suspicious products that were not successfully matched.
9. The method as described in claim 3, characterized in that, Also includes: Obtain information on abnormal shopping behavior identified by the shopping cart system; Get the shopping list sent by the shopping cart system; Analyze the correspondence between the matching results and the shopping list to obtain information on abnormal shopping lists; Send the abnormality list information and abnormal shopping behavior information to the verification personnel.
10. A supermarket merchandise damage prevention device, characterized in that, include: The first product recognition result acquisition module is used to obtain the first product recognition result set of the shopping cart system for the products in the shopping cart; The second product identification result acquisition module is used to obtain a second product identification result set of the smart loss prevention device on the products in the shopping cart after the shopping cart passes by. The first product identification result in the first product identification result set and the second product identification result in the second product identification result set both include the product category and the identification confidence level of each product category. The matching cost matrix calculation module is used to calculate the matching cost of each first product identification result and each second product identification result based on the product category and the identification confidence, forming a matching cost matrix; The matching module is used to match the first product identification result and the second product identification result according to the matching cost matrix to obtain the matching result, which includes successfully matched products and suspicious products that are not successfully matched.
11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.
13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.