Shopping cart inventory change indicator system

Abstract

A shopping cart system is disclosed which integrates multiple technologies for enhanced inventory management and theft detection. The system includes a barcode scanner, camera computer-vision system for real-time product recognition, a security scale to detect weight discrepancies, and an anomaly detection module for monitoring shopper behavior and identifying theft-related patterns. These components collectively improve inventory accuracy and security within retail environments.

Description

SHOPPING CART INVENTORY CHANGE INDICATOR SYSTEMTECHNICAL FIELD

[0001] The present disclosure pertains to the field of retail inventory management, specifically to systems and methods for integrating technological solutions with shopping cart self-checkout processes to address inventory shrinkage issues, including theft, misplacement, or mismatches, in grocery retail environments.BACKGROUND

[0002] The retail industry, particularly grocery stores, faces challenges associated with inventory shrinkage, including losses due to theft, damage, or administrative errors. These issues affect profitability, operational efficiency, and customer’s experience. Various technologies have been implemented to mitigate these losses and enhance the shopping experience, evolving towards more sophisticated solutions.

[0003] Inventory shrinkage is influenced by factors such as shoplifting, employee theft, supplier fraud, and administrative mistakes. Traditional methods, including physical security enhancements and procedural tactics, often do not fully resolve shrinkage and can lead to increased operational costs and negative effects on customer interactions. In response, the adoption of digital technology, such as Electronic Article Surveillance (EAS) systems including RFID and barcode scanning, has increased. These technologies facilitate item tracking and real-time inventory management, helping identify discrepancies and streamlining checkout processes.

[0004] The integration of Artificial Intelligence (Al) and computer vision has further advanced the capacity to address retail shrinkage. Al-powered systems analyze data to detect patterns and anomalies, while computer vision enables real-time inventory monitoring and the identification of suspicious behaviors. These technologies improve upon previous methods, offering more effective monitoring and detection capabilities.

[0005] However, there remains a need for a system that not only utilizes RFID and barcode scanning for inventory management but also incorporates the capabilities of Al and computer vision for a comprehensive approach to shrinkage prevention. Such a system should provide realtime, accurate monitoring and intervention capabilities, reducing shrinkage effectively while enhancing the shopping experience.SUMMARY

[0006] In one aspect, the present disclosure relates to a shopping cart system that integrates various technologies to facilitate inventory management and enhance security measures within a retail environment. The disclosed system includes a barcode scanner configured to decode barcodes of items for inventory tracking, a computer-vision system including camera to recognize products being added or removed from the cart, a security scale that measures item weights to identify discrepancies, and an anomaly detection module to monitor shopper behavior and detect theft- related anomalies.

[0007] In another aspect, the aforementioned shopping cart system may optionally include an RFID transceiver that enhances item tracking using RFID tag detection. This transceiver is configured to improve security for high-value items and streamline inventory management processes.

[0008] In yet another aspect, the camera of the shopping cart system’s computer vision technology is used to perform real-time detection and classification of items. This technology assists in accurately identifying the products added to or removed from the cart and verifying such actions against a cart inventory database.

[0009] In a further aspect, the shopping cart system is configured to communicate with a remote server via a network interface. This communication enables the transmission of data collected by the barcode scanner, RFID transceiver, computer-vision system including camera, and security scale for further processing and inventory management.

[0010] In an embodiment, the anomaly detection module of the shopping cart system utilizes deep learning models to improve the accuracy of detecting suspicious activities. This module assesses shopper behavior and correlates it with inventory changes to identify potential theft occurrences effectively.

[0011] In yet another embodiment, the shopping cart system can operate in multiple modes including IDLE, ADD ITEM, REMOVE ITEM, and UNEXPECTED WEIGHT, to adapt to different inventory changes and shopper actions within a retail setting. This adaptability enhances the system's responsiveness to various operational scenarios.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Fig. 1 shows a perspective view of a shopping cart system equipped with a barcode scanner, an RFID transceiver, a basket facing camera, and a security scale integrated into a conventional shopping cart structure.

[0013] Fig. 2 is security scale functional specification (FS), where the transition between states generates alerts and snapshots that can be used for notifications and for the algorithmic pipeline.

[0014] Fig. 3 the challenge of differentiating between items inside a shopping cart system marked in the green box, and items outside the shopping cart, marked by a red box.

[0015] Fig. 4A shows a shopping cart with two items, while Fig. 4B shows the addition of a cereal box (inside the green square).DETAILED DESCRIPTION

[0016] The present relates to a shopping cart system that enables and monitors self-checkout by the customer. The shopping cart includes a barcode scanner. The barcode scanner is employed to decode barcodes on shopping items, thus assisting in managing inventory and in the operation of self-checkout shopping carts (aka "smart shopping carts"). For instance, as items are added to or removed from a shopping cart, the barcode scanner can scan each item to ensure that the inventory records are updated in real time. This functionality aids retail stores in maintaining accurate stock levels and reduces the time required for inventory checks.

[0017] As used herein, the term 'barcode' refers broadly to any form of machine-readable marking, identifier, or code applied to or associated with a product, including but not limited to onedimensional barcodes, two-dimensional barcodes (e.g., QR codes), optical character recognition (OCR) markings, RFID tags, NFC tags, or any other optical, electronic, or magnetic markings that can be used to identify or retrieve information associated with a product, such as product name or price. For ease of reference and readability, the term 'barcode' is used throughout this application, but should be interpreted to encompass such broader marking technologies unless explicitly stated otherwise.

[0018] Additionally, the system incorporates a computer-vision system including camera. This camera is used for recognizing products as they are added to or taken out of the cart. The computervision system verifies the presence or removal of items by comparing them with a shop inventory database. For example, if an item is placed in the cart without scanning, the computer- visionsystem identifies the product via its camera and adds it to the inventory list displayed to the user, thereby enhancing the accuracy of both billing and inventory tracking. The computer-vision system may also alert the user that he forgot to scan the item and can present the user with a snapshot of the item.

[0019] The shopping cart system is also equipped with a security scale. This scale measures the weight of items in the cart and can identify discrepancies between the actual weight of the items and the expected weight based on the scanned barcodes. This feature is particularly useful in detecting errors in scanning or potential theft, as an incorrect weight can trigger an alert for further verification. For instance, if a shopper scans a lighter item but places a heavier item in the cart, either by mistake or on purpose, the discrepancy in weight will be detected by the security scale, and the system can react appropriately. For example, the system can cancel in the shopping list the last product entered, i.e., the lighter item, and then ask the user to remove the last product entered and rescan it in the system.

[0020] An anomaly detection module forms another integral component of the disclosed system. This module monitors the behavior of shoppers using various inputs, including data from the Al camera and the security scale, to detect abnormal patterns that may indicate theft. For example, if a shopper repeatedly removes items from the cart and walks away from checkout areas, the anomaly detection module can flag this behavior as suspicious, prompting further scrutiny.

[0021] Each component of the system works in concert to provide a robust solution for inventory management and loss prevention in retail environments. By integrating barcode scanning, visual recognition, weight measurement, and behavioral analysis, the system ensures that inventory is accurately tracked, and that discrepancies or abnormal behaviors are swiftly identified, thereby minimizing potential losses due to theft or scanning errors.

[0022] Preferably, but not mandatory, the shopping cart system can incorporate an RFID transceiver, equipped for identifying items placed within or removed from the cart. This RFID transceiver reads information from RFID tags attached to or embedded in the items. Through this process, each item within the shopping cart is recognized based on the encoded information on its respective RFID tag.

[0023] Furthermore, the RFID transceiver is configured to ascertain the location of the items within the cart. The precise determination of item placement is achieved by analyzing the signalstrength and orientation data received from the RFID tags. This functionality facilitates the tracking of item positions as they are added, rearranged, or removed from the cart.

[0024] Additionally, this system enhances the management and checkout processes by allowing for real-time item audits without requiring manual input or physical scanning of each item. By leveraging the capabilities of the RFID technology, the system streamlines the shopping experience, offering efficiencies in inventory management and customer service.

[0025] Fig. 1 shows an embodiment of a shopping cart of the invention equipped with various integrated technologies for inventory management and theft prevention. The elements labeled include barcode scanner 100, basket facing camera 110, security scale 120, and RFID transceiver 140, each installed on a standard shopping cart frame.

[0026] As illustrated, the barcode scanner 100 may be positioned at the handle of the cart, facilitating an accessible orientation for scanning merchandise as items are placed inside the cart. Basket facing camera 110 is located adjacent to the barcode scanner 100, with a direct view into the cart's basket, which enables real-time visual verification of items added or removed. The camera's alignment provides an optimal vantage point for monitoring cart contents and aids in discrepancy identification when cross-referencing with the cart's inventory database.

[0027] Additionally, security scale 120 is incorporated into, e.g., the bottom of the basket area, designed to detect variations in weight corresponding to the items scanned and added to the cart. This component plays a critical role in identifying when the actual weight does not match the expected weight based on the scanned items, thereby triggering alerts for potential unscanned or misrepresented items. Moreover, an optional RFID transceiver 140 can be mounted on the cart's handle, close to the basket facing camera 110 and barcode scanner 100. This positioning allows for enhanced detection and tracking of items embedded with RFID tags, which assists in securing high-value merchandise and ensuring accurate item identification within the cart.

[0028] These components collectively contribute to the system's functionality in preventing inventory shrinkage, facilitating efficient checkout processes, and enhancing overall inventory management through technological integration into the shopping cart's structure. Each component can be substituted or varied in different embodiments to accommodate different store layouts or to incorporate evolving technologies without deviating from the disclosed configurations.

[0029] The computer vision technology is used to execute the real-time detection of objects. This process involves the camera capturing visual data from its environment, which is then processedusing algorithms capable of identifying and distinguishing between different items within the view. For instance, in a retail setting, the camera might identify products inside or outside of the shopping cart.

[0030] Further, the system extends its capabilities to the classification of detected items. Through classification, the camera categorizes each item into predefined groups based on characteristics identified during the detection phase. For example, classifying products into categories such as beverage, cereal boxes and more.

[0031] The disclosed technology employs machine learning models that are trained using extensive datasets to progressively enhance accuracy. As the camera remains operational, it accumulates additional data, which is utilized to refine the algorithms and improve overall system performance. In the context of managing a shopping cart system, this capability enables the camera to more effectively differentiate between various types of items being placed into or removed from the cart, or items being rearranged in the cart (basket) such as distinguishing between packaged goods and fresh produce, thereby supporting accurate inventory management and checkout processes.

[0032] In security monitoring within a shopping environment, this technology illustrates its effectiveness by tracking activities around the shopping cart. The camera is equipped to identify unauthorized removal of items from the cart or unusual behavior near the cart area. It classifies these activities based on the potential risk they pose, allowing for immediate notifications to security personnel, which helps in preventing theft or misplacement of items.

[0033] Additionally, this Al-enhanced camera accommodates a variety of other functions relevant to the specific demands of the retail environment. For example, it can monitor the expiration dates of perishable goods in the shopping cart, identify when products are out of stock on the store shelves, or assess the weight of items placed in the cart to ensure that the correct product quantity is being purchased. This aids in providing critical information that facilitates efficient store management and enhances customer shopping experience.

[0034] Anomaly detection software monitors shopper behavior, flagging abnormal patterns that could signal suspicious activity, thereby enhancing overall security measures. The anomaly detection module employs Al algorithms to correlate observed shopper behavior with inventory changes. Examples of events monitored include:

[0035] Unattended Scans: a user may scan an item but then forget to place it in the shopping cart. To prevent charging the user for an item not in the cart, the security scale data is utilized to inhibit the next scan until the weight discrepancy between the actual and expected weights is resolved. This can be achieved by either placing the correct item into the basket or voiding the item. The Al camera system plays a crucial role by ensuring that the item placed in the cart corresponds accurately to the scanned item, thus maintaining the integrity of the transaction. Unscanned Items: the system is engineered to detect any items added to or removed from the shopping cart without being scanned. By cross-referencing RFID reports, weight changes, and snapshots taken before and after weight alterations, it's possible to generate alerts and provide visual aids to highlight discrepancies between the expected and actual items in the cart. For instance, if a shoplifter places an item into their shopping cart without scanning it, they might nonchalantly drop it in while pretending to browse. Since the item wasn't scanned, there's no corresponding entry to match against the weight recorded by the security scale. Consequently, the security scale is programmed to trigger an alert due to the unexpected weight increase. Furthermore, the Al camera system identifies the addition of the item to the cart without a corresponding scan, whilst the RFID can potentially identify the item, prompting an alert for staff to conduct further investigation. Barcode Switching: a shoplifter can employ a deception scanning the barcode of a low-value item while surreptitiously placing a high-value item into the basket. Should a weight disparity exist between the scanned item and the one introduced into the basket, the security scale will generate an alert. Nonetheless, in instances where the weights of the items align, such as two 750ml bottles of wine, the RFID tag read will discern the inconsistency between the scanned barcode and the item actually placed into the basket, thereby prompting an alert.

[0036] The security scale can be utilized as a measure to detect potential theft by implementing various strategies and technologies, including:

[0037] Threshold Alerts: set up a threshold weight limit for empty shopping carts, meaning a shopping cart where the user hasn't scanned any items thus the shopping cart is assumed to be empty. If the weight measured by the sensors exceeds this limit when a customer attempts to leave the store, it could trigger an alert. This could indicate that items have not been properly scanned and paid for. Compare Expected vs. Actual Weight: establish a baseline expected weight for some or all items based on their known weight or average weight. The system can compare the expected weight of scanned items in the shopping cart to the actual measured weight. Any significantdeviations could be flagged as potential discrepancies. Random Checks: Implement a system of random weight checks at the exit. Regardless of the weight measured during the shopping process, customers may be required to go through a random weight verification check at the exit to ensure that all items have been properly accounted for. RFID Input Integration: Combining weight sensor data with RFID or barcode scanning leads to a more accurate identification of the item in the shopping cart. Each item may have a unique identifier, and the system can cross-reference this information with the weight data. If an item without a corresponding scan is detected, it could be flagged as a potential issue. Camera Input Integration: Integrate weight sensor data with video surveillance systems. If a discrepancy is detected, the system can trigger video recording for further review by security personnel. Additionally, basic snapshots from the camera can be utilized to address unexpected weight events, which are inherent to the use of non-floating baskets and scales in smart carts. User Authentication: Implement user authentication mechanisms tied to the shopping cart. For example, customers may need to authenticate themselves at the beginning of their shopping trip, and the system can associate the weight measurements with their account. If discrepancies arise, it is easier to trace them back to specific individuals. Educational Measures: Display messages or alerts on the cart screen or through the store's app, reminding customers about the importance of scanning all items before leaving the store. This could serve as a deterrent and encourage compliance.

[0038] Shopping cart chassis-mounted security scales, may have either "floating" security scales or "non-floating" security scales. A "floating scale" allows for some movement or flexibility, beneficial for absorbing shocks or vibrations during cart movement to ensure accurate weight measurements. Conversely, a "non-floating scale" is securely bolted or rigidly attached to the chassis, providing stability but being more susceptible to inaccuracies from external factors like leaning or shocks. Within this context, "floating" and "non-floating" delineate the level of rigidity or flexibility in how the scale is mounted to the cart's chassis. A non-floating scale registers pressure whenever weight is applied to the cart, including external pressure like a shopper leaning on it.

[0039] "Non-floating" scales may be preferred due to cost considerations of hardware and assembly. To mitigate the effects of external forces on a smart cart equipped with a "non-floating" scale, such as leaning, specific methods can be employed. The system can sample weight at predefined intervals, for example, every 50 milliseconds, logging readings alongside reliabilityparameters provided by the scale. Consistency is ensured by considering a weight reading correct if several consecutive stable readings occur, for example, five consecutive readings, thus filtering out transient fluctuations or noise and enhancing accuracy. If, for example, 20 consecutive unstable samples of the same weight are collected, the system can consider this weight valid, prioritizing stability over the scale's inherent sensitivity to ensure reliability in recorded weights.

[0040] To address unexpected weight events comprehensively and distinguish between genuine additions to the cart and incidental contact, the system can rely on the camera system. Essentially, the camera is programmed to capture snapshots triggered by predefined events, primarily unexpected weight changes. Upon detecting such an event, the camera captures a snapshot, which is then compared to the last stable state of the cart. If no discernible change is detected in the captured image, indicating that the weight event was likely noise, the registered weight is disregarded.

[0041] The combination of barcode scans, a security weight, camera, and RFID scans, helps detect various inventory discrepancies during shopping or checkout processes. The following list exemplifies a few common use-cases where the combination of one or more security layer is used:

[0042] Adding an Unscanned Item - The system is designed to identify any items placed into the shopping cart without being scanned for purchase, by detecting an increase in weight and / or by an RFID scan. By comparing snapshots captured before and after weight changes or RFID scans, the system can generate alerts and visual aids to highlight differences between the expected (barcode scanned) and actual items in the cart. For instance, if a shopper places an item in their cart without scanning it, the system will flag this discrepancy. Missing a Scanned Item - The system is also designed to identify any items removed from the cart without being voided for checkout, by detecting an unexpected decrease in weight and / or RFID scan. By comparing snapshots captured before and after weight decreases, the system can trigger alerts and visual aids to highlight discrepancies between the expected (barcode scanned) and actual items in the cart. For example, if a shopper removes an item from their cart without scanning it out, the system will notify the shopper that he may be charged on an item that is not in the cart. Single Item Weight Discrepancy - Additionally, the system cross-references the weight of scanned items with their known weights and expected standard deviations. For instance, if item A, weighing 1000g with a known standard deviation of + / - 8g, has been scanned and item B, weighing 780g, is added to the cart, the system can alert on the weight discrepancy and prompt the shopper to rectify the situation or seekassistance. In rare cases where item A and item B have the same weight, the system will need to rely on RFID scans or computer- vision system’s camera to detect discrepancies. Multiple Items Weight Discrepancy - If a shopper scans an item and sets the quantity to four but adds five items to the cart, the system will flag this inconsistency, alerting the shopper to verify the quantity or prompting them to correct the discrepancy. In this case, managing the standard deviation of multiple items becomes important. To address this, the system calculates the total weight by multiplying the weight of a single item by the quantity scanned. Then, it combines the standard deviations using statistical principles. For identical items with the same standard deviation, the combined standard deviation is determined by multiplying the standard deviation of a single item by the square root of the quantity. After comparing the total weight and combined standard deviation to the expected values, any significant deviations trigger the system to flag the discrepancy. This prompts the shopper to verify the quantity scanned or take corrective action, such as adjusting the quantity or rescanning the items. Tampering with RFID Tags - While RFID technology effectively tracks high-value items, tampering with or removing tags can lead to discrepancies. In such cases, the system may detect differences between the expected RFID-tagged items and the actual inventory recorded by the computer- vision system’s camera and / or barcode scanner. This comparison is made against the reported weight from the security scale. For instance, if an RFID tag is manipulated or missing, the system may identify weight inconsistencies, prompting staff to conduct further investigation.

[0043] The system is designed to transition between the following states: IDLE, ADD ITEM, REMOVE ITEM, and UNEXPECTED WEIGHT. A snapshot is acquired as part of the transition between states. Generally, if the weight of an item is known, the system is expected to proceed accordingly. However, if it's a new item, the system weighs the item after scanning and updates its weight in the database for general use.

[0044] Fig. 2 shows a schematic representation of a shopping cart system highlighting operational states and transitions within an integrated security and inventory management system. This system diagram represents several key states including "IDLE", "ADD ITEM", "REMOVE ITEM", and "UNEXPECTED WEIGHT UNBALANCED", each associated with specific triggers and conditions that affect the system's behavior.

[0045] In the "IDLE" state, the cart awaits user interaction and is considered balanced. Actions that affect transition from this state include "scan item" and "add item", which move the systemto the "ADD ITEM" state, or adjustments in cart weight parameters (illustrated as "scale_weight_ok" and "scale_weight_override"), maintaining the system in the "IDLE" state. The "scale weight high / alert" condition triggers a transition from the "IDLE" state to the "UNEXPECTED WEIGHT UNBALANCED" state, indicating a need for immediate attention due to weight discrepancies.

[0046] In the IDLE state, the expected weight of the shopping cart matches the weight reported by the security scale. During the ADD ITEM state, an item (barcode) is scanned (scan item) or selected from the graphical menu of the cart (add item). At this point, there's a discrepancy between the expected weight of the cart and its actual physical weight.

[0047] When in the REMOVE ITEM state, the physical weight of the items in the cart is lower than the expected weight of the scanned inventory, or the user has chosen to void an item via the graphical interface of the cart. In both cases, the shopper is expected to scan the item before it's deleted from the cart's inventory. In the UNEXPECTED WEIGHT state, the physical weight of the items in the cart is higher than the expected weight of the scanned inventory. The user is expected to remove the excess item from the cart.

[0048] The "ADD ITEM" state involves adding items to the cart where "scale weight ok" and "scale weight override" allow the cart to return to or remain in the "IDLE" balanced state. Conversely, a "scale weight high / alert" results in a shift to the "UNEXPECTED WEIGHT UNBALANCED" state, signaling discrepancies between actual and expected weights. Similarly, in the "REMOVE ITEM" state, the conditions "scale_weight_low", "scale_weight_ok", and various overrides ("scale weight override") direct the workflow either back to the "IDLE" state or maintain in the "REMOVE ITEM" state, whereas "scale_weight_high / alert" pushes the system into the "UNEXPECTED WEIGHT UNBALANCED" condition.

[0049] The depiction of the "UNEXPECTED WEIGHT UNBALANCED" state is critical as it represents scenarios where the weight anomalies require rectification. This state is connected through transitions indicating either high or low alerts ("scale_weight_high / alert" and "scale weight low / alert"), and it connects back to the "IDLE" state upon resolution of these anomalies, either through overrides or corrections in weight measurements.

[0050] The schematic effectively demonstrates the dynamic response of the shopping cart system to various operational inputs and conditions, reflecting its capability to safeguard against inventoryshrinkage and ensuring accurate management of cart contents through advanced interaction between physical weight measurements and system-monitored activities.

[0051] The security scale initiates a snapshot each time it detects a potential alteration in the cart's weight. In addition to the security scale, the system can autonomously conduct RFID scans at predetermined intervals, for example, every 1 minute. Each scan operation concludes with the creation of a snapshot, which is then stored in system memory, such as an SD card along with its corresponding timestamp. Notably, the system compares each new snapshot with the previous one, retaining it only if variations are identified.

[0052] The process of monitoring changes in the cart’s inventory over time consists of several steps. Firstly, the Al-based, basket-facing camera system has to determine whether an item is inside or outside the shopping cart’s cage by using the cart’s cage as a reference grid. If the item occludes the expected grid (item in front of the grid), it is considered inside the cage. Conversely, if the grid occludes the item (grid in front of the item), it is considered outside the cart’s cage and will not be included in subsequent processes. Once the items of interest are isolated, the system seeks to identify changes in the observed landscape. If a change is detected, a snapshot is taken and saved with its timestamp for reference and future comparisons. These identified changes are marked with bounding boxes and forwarded to the next step, where the item’s cropped image is classified into predefined categories such as glass bottle, carton box, circular container, etc. The classification is compared against the last scanned item, weight change, RFID indication, and any existing discrepancies.

[0053] The final stage involves item recognition. An item (product) catalogue is inputted into a real-time detection network. This network, based on the YOLO (You Only Look Once) model, is trained and tested with the input catalogue items. The model can accurately identify the actual item and differentiate between similar-looking items, even ones that weigh the same and look almost identical, such as cheap and expensive bottles of wine.

[0054] The following steps describe the implementation of the cart inventory change detection & recognition method. The first two steps are considered mandatory and are basically used as noise filters. Steps 3 and 4 are optional and can enhance the overall result with very specific information.

[0055] Cart Grid Detector - To determine whether an item is inside the cart’s cage or not, the system employs SegFormer, a simple, efficient, and powerful semantic segmentation method. This approach helps accurately identify the spatial boundaries of the cart's cage in images, allowing forprecise localization of items within or outside the designated area. Cart Inventory Change- To track the items inside the cart’s cage and identify any changes, the system utilizes Changer as a foundational approach for this task. Changer provides a basic framework for detecting alterations in the observed inventory landscape. Item Classification - For item classification, a convolutional neural network (CNN) is one suitable choice. CNN's are widely used in image recognition tasks and have demonstrated excellent performance in classifying objects within images. Specifically, architectures like ResNet (Residual Neural Network), VGG (Visual Geometry Group), or InceptionNet can be employed for this purpose. These networks are pretrained on large image datasets like ImageNet and can be fine-tuned on the specific dataset of item images to improve classification accuracy. Additionally, transfer learning techniques can be utilized to leverage the knowledge learned from the pretrained models and adapt it to the item classification task. Item Recognition - For item recognition, a popular choice is a convolutional neural network (CNN) due to its effectiveness in image recognition tasks. Specifically, one could use architectures like YOLO (You Only Look Once), SSD (Single Shot Multibox Detector), or Faster R-CNN (Region-based Convolutional Neural Network). These networks are designed to detect and classify objects within images accurately and efficiently. YOLO, in particular, is well-suited for real-time object detection tasks, making it a suitable option for item recognition in a dynamic environment like a shopping cart. Additionally, these networks can be fine-tuned on the specific dataset of item images to improve recognition performance.

[0056] Fig. 3 illustrates the challenge of differentiating between items inside a shopping cart system marked in the green box, and items outside the shopping cart, marked by a red box.

[0057] Fig. 3 displays items that are located within a shopping cart system, these items are delineated by a boundary depicted in a green coloration. The boundary aids in distinguishing these items from others that are not part of the shopping cart system.

[0058] As depicted in Fig. 3, without preventive measures, the item in the adjacent cart (highlighted with a red box) may erroneously be regarded as part of the cart’s inventory (highlighted in green). To mitigate this issue, the system leverages the cart’s aluminum cage, which can serve as a grid-like structure. It is evident that the item inside the cart obscures the grid, while the item in the adjacent cart is obscured by the aluminum cage of the cart itself. This distinction allows the system to accurately determine which items belong to the shopping cart’s inventory andwhich do not, thereby ensuring the integrity of the inventory monitoring system, making sure clients are charged only for the products in their shopping carts.

[0059] Semantic Segmentation:

[0060] SegFormer is a transformer-based framework designed for semantic segmentation tasks, focusing on efficiency, accuracy, and robustness. Unlike previous methods, SegFormer reimagines both the encoder and decoder components, offering several key innovations:

[0061] Transformer encoder - this encoder design avoids the need for interpolating positional codes during inference, allowing the model to adapt easily to test resolutions different from the training resolution. Additionally, the hierarchical structure enables the encoder to generate both high-resolution fine features and low-resolution coarse features, providing more flexibility compared to previous models like ViT. MLP Decoder - A lightweight MLP decoder that leverages Transformer-induced features. By exploiting the attention patterns of different layers in the Transformer, the decoder combines both local and global attention, resulting in a simple yet powerful representation. This approach aggregates information from multiple layers to achieve better performance, without relying on complex and computationally demanding modules.

[0062] Detecting changes in a shopping cart's inventory is essential for maintaining accurate tracking over time. Traditionally, this task resembles tracking tasks, wherein each item in the cart must be uniquely identified and monitored over time. However, treating the content of the basket as a landscape and considering additions or subtractions of items as changes in the landscape topology offers a more effective approach. This perspective allows the system to detect changes in the cart's inventory without the need to individually track each item over time, streamlining the process and improving efficiency. While conventional methods often overlook the interactions between items in cart images, this new approach considers how these items interact over time, enabling more comprehensive and accurate inventory change detection.

[0063] Figs. 4A and 4B illustrate the detection of the addition of a Kellogg’s cereal box. Fig. 4A shows an earlier snapshot of a shopping cart with two items inside. In the snapshot of Fig. 4B, an additional item is observed, a Kellogg’s cereal box, shown inside the green box. For identification of the added item the system verifies the results of the barcode reader, the results of the RFID tag reader and possible weight changes, all tracked over time.

[0064] In one embodiment, the system follows the suggested models, Changer AD and ChangerEx, that use basic strategies. ChangerAD simplifies complex methods, while ChangerEx allows foreasy swapping of information between the images. Additionally, the Flow Dual- Alignment Fusion (FDAF) module cab be used to improve the alignment and merging of items in the images.Item Classification

[0065] Cart inventory item classification using ResNet involves leveraging the Residual Neural Network (ResNet) architecture to classify items within the shopping cart. First, a dataset of labelled images containing various items is prepared. These images are preprocessed and then used to train the ResNet model through techniques like transfer learning. During training, ResNet learns to extract relevant features from the input images and classify them into different categories. Once trained, the model can accurately classify new images of cart inventory items in real-time. This enables automated inventory management and tracking, enhancing operational efficiency in retail environments. Good classification categories for expensive items in a supermarket, suitable for ResNet, could include: (Fine Wines, Premium Spirits, Luxury Skincare, Gourmet Foods, High- end Electronics, Designer Apparel, Specialty, Kitchen Appliances, Artisanal Delicacies, Exclusive Homeware, Luxury Confectionery). These categories encompass high-value items typically found in upscale supermarkets or specialty stores. Leveraging ResNet for classifying such items can help retailers effectively manage and safeguard their valuable inventory.Item Recognition

[0066] Cart inventory items' recognition using YOLO (You Only Look Once) entails leveraging the YOLO object detection framework to swiftly and accurately identify items within the shopping cart. This involves training the YOLO model on a dataset containing annotated images of various cart inventory items, enabling it to recognize and localize different types of items in real-time. The YOLO architecture efficiently predicts bounding boxes and class probabilities for each item in a single pass through the network, making it highly effective for object detection tasks.

[0067] Once trained, the YOLO model can detect multiple items within the shopping cart simultaneously, providing bounding box predictions and corresponding class probabilities for each detected item. Post-processing techniques such as non-maximum suppression (NMS) are then applied to refine the final set of detected items and eliminate redundant bounding boxes. Optionally, a separate classification step can be performed to classify the detected items into predefined categories, further enhancing the model's capabilities for inventory management andretail applications. Deployed in a production environment, YOLO facilitates automated inventory management, theft prevention, and other real-time applications in retail settings.

[0068] The shopping cart system is equipped with a network interface configured to transmit data to a remote server. This data originates from several sources attached to or integrated within the shopping cart, including a barcode scanner, an RFID transceiver, a computer-vision system including camera, and a security scale. The transmission of this data facilitates subsequent processing steps that are performed remotely.

[0069] Upon collection, the data from the barcode scanner, RFID transceiver, camera, and security scale is conveyed through the network interface. The interface ensures the timely and efficient delivery of the collected data to a designated remote server. This server is configured to receive and process the incoming data for various operational needs.

[0070] The processed data on the remote server supports critical functions such as inventory management. By utilizing the transmitted information from the shopping cart system, inventory levels can be updated in real-time. This real-time update helps maintain accurate inventory tracking and management, thereby enhancing the efficiency of inventory control systems within a retail environment.

[0071] Although the invention has been described in detail, changes and modifications, which do not depart from the teachings of the present invention, will nevertheless be evident to those skilled in the art. Such changes and modifications are deemed to come within the purview of the present invention and the appended claims.

[0072] It will be readily apparent that the various methods and algorithms described herein may be implemented by, e.g., appropriately programmed general purpose computers and computing devices. Typically, a processor (e.g., one or more microprocessors) will receive instructions from a memory or like device, and execute those instructions, thereby performing one or more processes defined by those instructions. Further, programs that implement such methods and algorithms may be stored and transmitted using a variety of media in various ways. In some embodiments, hardwired circuitry or custom hardware may be used in place of, or in combination with, software instructions for implementation of the processes of various embodiments. Thus, embodiments are not limited to any specific combination of hardware and software.

[0073] A "processor" means any one or more microprocessors, central processing units (CPUs), computing devices, microcontrollers, digital signal processors, or like devices.

[0074] The term "computer-readable medium" refers to any medium that participates in providing data (e.g., instructions) which may be read by a computer, a processor or a like device. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks and other persistent memory. Volatile media include dynamic random-access memory (DRAM), which typically constitutes the main memory. Transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise a system bus coupled to the processor. Transmission media may include or convey acoustic waves, light waves and electromagnetic emissions, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

[0075] Various forms of computer readable media may be involved in carrying sequences of instructions to a processor. For example, sequences of instruction (i) may be delivered from RAM to a processor, (ii) may be carried over a wireless transmission medium, and / or (iii) may be formatted according to numerous formats, standards or protocols, such as Bluetooth, TDMA, CDMA, 3G.

[0076] Where databases are described, it will be understood by one of ordinary skill in the art that (i) alternative database structures to those described may be readily employed, and (ii) other memory structures besides databases may be readily employed. Any illustrations or descriptions of any sample databases presented herein are illustrative arrangements for stored representations of information. Any number of other arrangements may be employed besides those suggested by, e.g., tables illustrated in drawings or elsewhere. Similarly, any illustrated entries of the databases represent exemplary information only; one of ordinary skill in the art will understand that the number and content of the entries can be different from those described herein. Further, despite any depiction of the databases as tables, other formats (including relational databases, object-based models and / or distributed databases) could be used to store and manipulate the data types described herein. Likewise, object methods or behaviors of a database can be used to implement variousprocesses, such as the described herein. In addition, the databases may, in a known manner, be stored locally or remotely from a device which accesses data in such a database.

[0077] The present invention can be configured to work in a network environment including a computer that is in communication, via a communications network, with one or more devices. The computer may communicate with the devices directly or indirectly, via a wired or wireless medium such as the Internet, LAN, WAN or Ethernet, Token Ring, or via any appropriate communications means or combination of communications means. Each of the devices may comprise computers, such as those based on the Intel.RTM. Pentium. RTM. or Centrino.TM. processor, that are adapted to communicate with the computer. Any number and type of machines may be in communication with the computer.

Claims

Claims1. A shopping cart system comprising: a barcode scanner configured to decode a barcode of an item and facilitate inventory management; computer-vision system comprising a camera and configured to recognize items added or removed from the cart and verify these against a cart inventory database; a security scale configured to measure weight of items and recognize discrepancies between actual weight and expected weight of scanned items; and an anomaly detection module configured to monitor shopper behavior and detect abnormal patterns indicative of theft by employing one or more Al algorithms to correlate observed shopper behavior with inventory changes.

2. The shopping cart system of claim 1, further comprising an RFID transceiver configured to identify and locate items within the cart utilizing RFID tags.

3. The shopping cart system of claim 2, wherein the RFID transceiver tracks high-value items with embedded RFID tags for enhancing security measures.

4. The shopping cart system of claim 1, wherein the computer-vision system performs realtime detection, classification, feature extraction and tracking of items.

5. The shopping cart system of claim 4, further comprising identification of said items.

6. The shopping cart system of any one of claims 1 to 5, wherein item classification and recognition employ models including ResNet, YOLO, SSD, and Faster R-CNN for categorizing, feature extraction, tracking and recognizing items.

7. The shopping cart system of any one of claims 1 to 6, further comprising a network interface to transmit data collected by the barcode scanner, RFID transceiver, computer -vision system, and security scale to a remote server for processing and inventory management.

8. The shopping cart system of claim 1, wherein the security scale triggers alerts upon detecting discrepancies between scanned items’ expected weight and actual weight data measured by the security scale.

9. The shopping cart system of claim 1, wherein the anomaly detection module utilizes deep learning models to enhance detection of suspicious activities.

10. The shopping cart system of claim 1, capable of transitioning between various operational modes including IDLE, ADD ITEM, REMOVE ITEM, and UNEXPECTED WEIGHT, based on detected changes in cart inventory and shopper actions.

11. The shopping cart system of claim 1 , further comprising a semantic segmentation module for determining boundaries of a cart's cage and differentiating between items inside or outside the cart.

12. A method for managing inventory in a shopping cart system, the method comprising: scanning a barcode of an item using a barcode scanner; recognizing items added or removed from the cart using a computer-vision system comprising a camera and verifying these against a cart inventory database; measuring the weight of added items using a security scale and recognizing discrepancies between the actual weight and expected weight of scanned items; and monitoring shopper behavior using an anomaly detection module to detect abnormal patterns indicative of theft by employing one or more Al algorithms to correlate observed shopper behavior with inventory changes.

13. The method of claim 12, further comprising identifying and locating items within the cart using an RFID transceiver that utilizes RFID tags.

14. The method of claim 13, wherein the RFID transceiver tracks high-value items with embedded RFID tags to enhance security measures.

15. The method of claim 12, wherein recognizing products includes real-time detection, classification, feature extraction and tracking of items.

16. The method of claim 15, further comprising identification of said items.

17. The method of any one of claims 12 to 16, further comprising transmitting data collected from the barcode scanner, RFID transceiver, computer-vision system, and security scale to a remote server for processing and inventory management via a network interface.

18. The method of claim 12, wherein measuring the weight of items triggers alerts upon detecting discrepancies between scanned items’ expected weight and actual weight data.

19. The method of claim 12, wherein the anomaly detection module utilizes deep learning models to enhance detection of suspicious activities.

20. The method of claim 12, further comprising transitioning the shopping cart system between various operational modes including IDLE, ADD ITEM, REMOVE ITEM, and UNEXPECTED WEIGHT, based on detected changes in cart inventory and shopper actions.

21. The method of claim 12, wherein recognizing and distinguishing items inside or outside the cart involves employing a semantic segmentation module for determining boundaries of a cart's cage.

Images