Fraud prevention and surface acoustic system

Abstract

To provide a method and device for fraud detection.SOLUTION: Acoustic waves are emitted towards a set of items in a receptacle, and sensor data are received from one or more acoustic wave sensors that receive reflected acoustic waves from the set of items. A model of the set of items in the receptacle is generated based on the sensor data. One or more features of the set of items are identified by analyzing the model. Checkout data are retrieved from one or more checkout devices. The one or more features of the set of items identified from the model are compared with the checkout data. When a mismatch between the one or more features and the checkout data is detected, an alert is issued.SELECTED DRAWING: Figure 1

Description

Background Art

[0001]

[0001] Acoustic wave sensors can be used to detect and analyze various physical properties of objects over a wide range of applications. These sensors operate by emitting acoustic waves and then analyzing the signals that return from nearby objects. Through this process, the size and shape of nearby objects can be effectively mapped, providing valuable information about the surrounding environment. In addition, by emitting and collecting acoustic waves, acoustic wave sensors can further provide depth-related information such as time-of-flight that can be used to measure the distance of an object from the sensor. This function is useful in scenarios where understanding spatial relationships is important for determining the position and number of objects within a given space. The accuracy and reliability of acoustic wave sensors when capturing and analyzing acoustic signals, combined with their ability to provide a wide range of characteristic information, have led to the widespread use of these sensors in environmental detection and object identification.

Brief Description of the Drawings

[0002]

Figure 1

[0002] FIG. 1 is a diagram showing an exemplary environment for advanced fraud detection using acoustic technology according to some embodiments of the present disclosure.

Figure 2

[0003] FIG. 2 is a diagram showing the propagation paths of acoustic waves induced by acoustic wave sensors installed at various positions around a container according to some embodiments of the present disclosure.

Figure 3

[0004] FIG. 3 is a diagram showing an exemplary workflow for detecting checkout fraud using sensor data according to some embodiments of the present disclosure.

Figure 4

[0005] FIG. 4 is a diagram showing an exemplary method for issuing a warning in response to potential checkout fraud based on collected sensor data and checkout data according to some embodiments of the present disclosure.

Figure 5

[0006] FIG. 5 is a flowchart showing an exemplary method for fraud detection according to some embodiments of the present disclosure.

Figure 6

[0007] FIG. 6 is a diagram showing an exemplary computing device configured to implement various aspects of the present disclosure according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0003]

[0008] For ease of understanding, whenever possible, the same reference numbers are used to designate the same elements common to each figure. It is contemplated that elements disclosed in one embodiment may be beneficially used in other embodiments even if not described.

[0004]

[0009] In some embodiments of the present disclosure, a scan area is provided near a self-checkout queue at a business site (e.g., a retail facility). Within the scan area, one or more acoustic wave sensors are installed to capture the contents of a cart or other receptacle (e.g., a basket, a bag) as a waiting customer passes through the area. In some cases, "receptacle" may refer to any container that can carry or store items, which may be open on one or more sides or completely sealed. This includes, but is not limited to, carts, baskets, and bags. Additionally, in situations where items are simply carried by the user's hand or arm, the user himself or herself may be considered a receptacle. These sensors may be placed at various positions within the scan area to capture the contents of the receptacle from various angles. The sensors operate by emitting acoustic waves towards the receptacle and then receiving signals that have been reflected, distorted, or otherwise changed by the receptacle or the items within it. In some embodiments, these received signals are then aggregated and analyzed to generate a model (e.g., a 3D representation) indicative of the contents of the receptacle. In some embodiments, the model may detail the physical dimensions, shapes, and arrangements of the items within the receptacle.

[0005]

[0010] In some embodiments of the present disclosure, the model generated based on sensor data can be further analyzed to identify various properties or characteristics of the items within the container. This analysis can include determining the total quantity of items within the container, as well as the shape, size, position, material, or other relevant characteristics of each item. In some embodiments, after extracting these features, more advanced analysis such as object recognition can be performed using a trained machine learning (ML) model. The results of the advanced analysis can include a list of identified items classified based on the features of the items extracted from the sensor data.

[0006]

[0011] In some embodiments, the extracted item features such as the identified quantity of items within the container, and / or the results of the advanced analysis such as the list of identified items can then be compared to the corresponding checkout data. In some embodiments, the checkout data can be provided by a checkout device at the business site. By cross-referencing the items detected by the sensor with those scanned by the checkout device, the system can effectively determine discrepancies that may indicate fraud. For example, if an item detected within the container has not been scanned by the checkout device, or if there is an inconsistency in the size or quantity of the item, the system can flag these transactions as potential fraud and alert the store clerk to take immediate action (e.g., manual verification). The disclosed method that includes using an acoustic sensor to detect items within the container and comparing the results to the checkout record not only streamlines the self-checkout process but also improves the detection of fraudulent checkout behavior.

[0007]

[0012] FIG. 1 shows an exemplary environment 100 for advanced fraud detection using acoustic technology, according to some embodiments of the present disclosure.

[0008]

[0013] In some embodiments, the environment 100 for fraud detection may correspond to a retail establishment (e.g., a grocery store, supermarket, outlet mall, or membership warehouse store). In the illustrated example, the environment 100 includes one or more self-checkout devices 110, one or more acoustic wave sensors 115, one or more servers 135, and a database 140. In some embodiments, one or more of the illustrated devices may be physical devices or systems. In other embodiments, one or more of the illustrated devices may be implemented using virtual devices and / or span multiple devices.

[0009]

[0014] In the illustrated example, the self-checkout devices 110, acoustic wave sensors 115, servers 135, and database 140 are located far apart from each other and communicate with each other via the network 130. Each of the devices may be implemented using an individual hardware system. The network 130 may include, correspond to, or be any combination of a wide area network (WAN), local area network (LAN), the Internet, an intranet, or any suitable communication medium available, and may include wired links, wireless links, or a combination of wired and wireless links. In some embodiments, each of the devices may be local to each other (e.g., within the same local network and / or the same hardware system), and may communicate with each other using any suitable local communication medium, such as a local area network (LAN) including a wireless local area network (WLAN), a hardwire, a wireless link, or an intranet.

[0010]

[0015] In the illustrated example, a plurality of self-checkout devices 110 are arranged within a self-checkout area 105 where a customer can scan, pay for, and / or bag their selected items by themselves. Each self-checkout device 110 may include various components including, but not limited to, a monitor, a barcode scanner, a weighing scale, a payment terminal, and a printer. The components operate together to ensure a smooth and efficient self-checkout experience for the customer. As shown, there is a self-checkout queue 120 near the self-checkout area 105 where customers line up with their containers 125 and wait their turn for self-checkout. As described above, "container" can refer to any container (which may have one or more open sides or be completely sealed) including, but not limited to, a cart, a basket, or a bag that can carry or store items. Additionally, in embodiments where items are simply carried by the user's hand or arm, the user themselves can be considered a container.

[0011]

[0016] In the illustrated example, a scan area 145 is provided within or near the self-checkout queue 120. In the illustrated example, the scan area 145 includes a gate-like structure 112 with one or more acoustic sensors 115 installed around it. The acoustic sensors 115 are arranged on various sides of the gate 112 such as the left side (e.g., 115-4) and the right side (e.g., 115-2), as well as the top side (e.g., 115-1) and the bottom side (e.g., 115-3) as shown. When a shopping cart 125-2 or another type of container (e.g., a basket or a bag) passes through the gate 112, the sensors 115 scan the cart and its contents from multiple angles. The data collected by the various sensors can be aggregated and used to generate a spatial model (e.g., a 3D model) representing the contents of the cart.

[0012]

[0017] In some embodiments, instead of using the gate-like structure 112, the scan area 145 may be an open space near or within the checkout queue 120. In such a configuration, the acoustic wave sensor 115 can be attached to various surfaces such as the ceiling, walls, and / or floor of the open area. When a customer moves their container through this scan area, the installed acoustic wave sensor 115 can be activated to scan the contents of the container. This configuration provides greater flexibility in terms of space and can be easily integrated into an existing layout.

[0013]

[0018] An illustrative example showing four acoustic wave sensors 115 installed in the scan area 145 is presented for conceptual clarity. In some embodiments, any number of acoustic wave sensors 115 can be installed at various positions within the scan area to capture a comprehensive view of the contents of the container as the contents pass through.

[0014]

[0019] In the illustrated example, each acoustic wave sensor 115 emits acoustic waves towards cart 125-2 and receives signals reflected by cart 125-2 or the items therein. Characteristics such as the time of flight (also called time delay in some embodiments), amplitude, and frequency change of these emitted and reflected waves can provide valuable information regarding the size, shape, or material composition of the items within cart 125-2. The received signals can then be transmitted to central server 135 for further processing and analysis. In some embodiments, acoustic wave sensor 115 can include a built-in processor (e.g., 205 of FIG. 2), which enables the sensor to preprocess the received signals before transmitting the signals to central server 135. The preprocessing can include filtering the signals to remove noise or irrelevant frequencies, or performing initial data analysis (e.g., identifying the time of flight, frequency change, or other relevant parameters). The preprocessing by acoustic wave sensor 115 facilitates a distributed processing architecture where a portion of the data analysis is performed at the sensor level. Such a configuration effectively reduces the computational load on central server 135 and thus optimizes the overall performance of the system.

[0015]

[0020] In the illustrated example, upon receiving data transmitted by the acoustic wave sensor 115 (e.g., information regarding the transmitted and reflected waves and processed parameters), the central server 135 may interpret the data and perform various analyses. In some embodiments, the server 135 may construct a visual representation of the contents of the cart based on the sensor data. In some embodiments, the visual representation may include a three-dimensional (3D) model showing the physical dimensions, shapes, and arrangements of the items within the cart 125-2. Further, in some embodiments, the server 135 may extract features or characteristics of these items based on the generated model and the collected sensor data. These features may include the total quantity of items within the cart 125-2, as well as details such as the size, shape, or material of each item. In some embodiments, item-specific features extracted from the sensor data and / or the 3D model may be used for accurate object recognition using a trained machine learning algorithm. Object recognition may enable the server 135 to identify individual items within the cart 125-2 and generate a corresponding list. In some embodiments, the object recognition process may include comparing the extracted features to a database of features learned from known items to ensure accurate identification / classification. In some embodiments, the generated list of items (which may also be referred to as a list of identification information in some embodiments) may distinguish individual items such as one bottle of water or two packages of cereal. In some embodiments, the list may classify items into broader categories such as "beverage in one bottle" and "two snack foods".

[0016]

[0021] In some embodiments, following feature extraction and / or object recognition, server 135 may perform a comparison process to detect fraud. In some embodiments, the comparison may include a simple quantity check, in which case the quantity of items identified from the 3D model is compared to the quantity scanned from corresponding checkout data (e.g., provided by a self-checkout device at a business site). In some embodiments, the comparison may include item-by-item or class-by-class verification and may be more complex. In such a configuration, server 135 may compare a list of items generated using object recognition techniques to a list of items actually scanned at the self-checkout device (e.g., identified from checkout data). Based on this comparison, server 135 may identify discrepancies between the two lists, such as unscanned items or quantity mismatches.

[0017]

[0022] In some embodiments, when any discrepancies are detected, server 135 may issue a warning. In some embodiments, server 135 may send a warning to a store clerk or customer for further action such as re-scanning an item, manually verifying the contents of the shopping cart, or resetting the self-checkout device if the discrepancy was caused by a potential problem within the device. In some embodiments, discrepancies detected between two lists, such as a list generated using object recognition technology and a list of items actually scanned at the self-checkout device, may not necessarily indicate fraud. Rather, such discrepancies may be caused by simple user error. For example, a customer may accidentally scan an item multiple times at a self-checkout device (e.g., 110-1) when they only intended to purchase one of that item. In such a configuration, server 135 may instruct the self-checkout device (e.g., 110-1) to display a message on its screen / interface to notify the customer of the discrepancy and / or suggest that the customer re-scan the items in their cart. Thus, the discrepancy detection mechanism can prevent inadvertent customer losses by ensuring that the customer is only charged for the items they actually intend to purchase.

[0018]

[0023] In the illustrated example, database 140 is configured to store data collected from various sources and / or data generated during various analyses. For example, the database may include sensor data collected by acoustic wave sensor 115 (in scan area 145), checkout data from self-checkout device 110, generated visual representations (e.g., 3D models) and their corresponding features, and comparison results (e.g., whether any discrepancies were detected). The stored data is then used to identify patterns or recurring problems, and thus the overall performance of the fraud detection system can be improved.

[0019]

[0024] FIG. 2 shows the propagation paths of acoustic waves induced by acoustic wave sensors installed at various positions around the container according to some embodiments of the present disclosure.

[0020]

[0025] The figure provides a top view of the scan area (e.g., 145 in FIG. 1). For example, the acoustic wave sensor 115-4 is shown as being installed on the left side of the scan area, and the acoustic wave sensor 115-2 is shown as being installed on the right side of the scan area. The two acoustic wave sensors 115-2 and 115-4 may be attached to a wall or to a gate-like structure (e.g., 112 in FIG. 1). As shown, each acoustic wave sensor (e.g., 115-4) includes a transmitter (e.g., 210-4), a receiver (e.g., 215-4), and an integrated processor (e.g., 205-4).

[0021]

[0026] When a shopping cart 125-2 or another type of receptacle (e.g., a basket or bag) is passed through the scan area (e.g., by a customer waiting in a self-checkout line), the transmitter 210-4 of the acoustic wave sensor 115-4 emits an acoustic wave 220a towards the left side of the cart 125-2. The emitted wave 220a can strike an item 225-1 within the cart 125-2 and then be reflected back towards the sensor 115-4. The reflected wave 220b is then collected by the receiver 215-4 of the sensor 115-4. The reflected wave 220b, in combination with the emitted wave 220a, can provide valuable information regarding the left side of the contents of the cart. For example, the time of flight can be calculated by measuring the duration between the time when wave 220a is emitted and the time when wave 220b is received. In this case, the time of flight data can be used to determine the distance of the item 225 from the sensor 115-4. In some embodiments, such as when one or more items are stacked on top of each other within the cart, depth information (e.g., the distance between the item and the sensor) can enable the system to accurately identify the spatial arrangement of the items within the cart. By understanding the arrangement and stacking of the items, the system can more accurately assess the contents of the cart, such as the total number of items within the cart and their individual physical attributes (e.g., size or shape). In some embodiments, the intensity (or amplitude) of the reflected wave 220b can provide information regarding the size, shape, and / or material of an item (e.g., 225-1) within the cart. In some embodiments, changes in the frequency of these waves (e.g., the reflected wave 220b and the emitted wave 220a) can be used to detect movement within the cart, such as item displacement or dropping, which can help in more accurately assessing the items within the cart.

[0022]

[0027] When shopping cart 125-2 passes through the scan area, the right acoustic wave sensor 115-2 (which operates simultaneously with, or sequentially after, the left sensor 115-4) collects data regarding the right side of cart 125-2. The acoustic wave sensor 115-2 emits an acoustic wave 220c (by transmitter 210-2) towards cart 125-2 and receives the acoustic wave 220d reflected by cart 125-2 or its items 225-2 (by receiver 215-2). The processing in the right acoustic wave sensor 1115-2 is similar to that of the left acoustic wave sensor 115-4, where the wave interacts with the contents of the cart and returns to the sensor.

[0023]

[0028] In the figure, sensors 115-4 and 115-2 have built-in computing capabilities by integrated processors 205-4 and 205-2. Such a configuration can enable sensors 115-4 and 115-2 to perform preliminary analysis locally, such as processing wave data (e.g., emitted waves 220a and 220c, and / or reflected waves 220b and 220d) to identify relevant parameters (e.g., time of flight, amplitude, frequency change). In some embodiments, the data collected by acoustic wave sensors 115-4 and 115-2 can be directly transmitted to a central server system (e.g., 135 in FIG. 1) for further processing and analysis.

[0024]

[0029] The two acoustic wave sensors 115-4 and 115-2 shown in the figure are presented for conceptual clarity. As described above, in some embodiments, any number of sensors 115 can be installed in the scan area to optimize the detection process. These sensors can be installed at various positions, such as the lower, upper, left, or right side of the scan area, or at the corners, to capture the contents of cart 125-2 from various angles. Such a sensor arrangement can ensure that a more comprehensive scan range properly captures all parts of cart 125-2, especially for items that may be covered or hidden under other items.

[0025]

[0030] FIG. 3 shows an example of a workflow 300 for detecting checkout fraud using sensor data according to some embodiments of the present disclosure. In some embodiments, the workflow 300 may be implemented by one or more computing systems, such as the server 135 shown in FIG. 1 and / or the computing device 600 shown in FIG. 6.

[0026]

[0031] In the illustrated example, the sensor data 305 is provided to a model generation component 310 for processing. In some embodiments, the sensor data 305 may refer to information collected by an acoustic wave sensor (e.g., 115 of FIG. 1), as the acoustic wave sensor radiates waves towards the container and its contents and receives the reflected signals from the container and its contents. The sensor data may include raw wave data such as details of the radiated and received waves, and data derived from these waves, such as time of flight (or time delay), amplitude of the reflected signal, frequency change of the wave when it returns, and the like.

[0027]

[0032] In the illustrated example, upon receiving the sensor data 305, the model generation component 310 analyzes the data to generate a visual representation of the contents of the container. In some embodiments, the visual representation may include a 3D model 315. The 3D model can show the contours of the items (e.g., 225 of FIG. 2) within the container and their spatial arrangement. In some embodiments, the contours of these items (e.g., 225 of FIG. 2) may be determined based on changing characteristics of the acoustic waves, such as the intensity (or amplitude) of the reflected signal that provides valuable information about the shape and size of these items, or the frequency change when they return. In some embodiments, the spatial arrangement of these items within the container may be determined based on depth information calculated from time of flight data that measures the distance between the items and the sensor by tracking how long it takes for the radiated wave to be reflected and return to the sensor.

[0028]

[0033] In the illustrated example, the generated 3D model 315 is then provided to the feature extraction component 320. In some embodiments, the feature extraction component 320 may be configured to analyze the 3D model 315 and extract various features or characteristics 325 of the items within the container. For example, in some embodiments, the feature extraction component 320 may evaluate the total quantity of items within the container by analyzing the number of distinct items within the 3D model 315. In some embodiments, based on the depth information and contour information available in the 3D model 315, the feature extraction component 320 may identify the size and / or shape of each item. In some embodiments, various materials may affect the received acoustic signal differently, resulting in various acoustic signatures. For example, metal may strongly reflect acoustic waves, while plastic may absorb some of the wave energy, resulting in various acoustic patterns or signatures. Such differences can be captured and incorporated into the 3D model. In some embodiments, the feature extraction component 320 may further infer the material properties of the items based on the acoustic signatures presented in the 3D model. As illustrated, the output of the feature extraction component 320 includes feature data 325 that includes, but is not limited to, the total quantity of items within the container, the physical arrangement of these items (e.g., how the items are stacked or placed relative to each other), and the physical attributes of each item (e.g., size, shape, or material), which are various features extracted from the 3D model 315.

[0029]

[0034] In the illustrated example, the feature data 325 extracted from the 3D model 315 may be sent directly to the fraud detection component 340 to detect the occurrence of fraud during the self-checkout process and / or sent to the object recognition component 330 to further identify or classify each item within the container before performing fraud detection. As illustrated, the checkout data 345 is also provided to the fraud detection component 340. In some embodiments, the checkout data 345 is taken from a self-checkout device (e.g., 110 of FIG. 1) at the business location and may include transaction information for verifying the list of items actually scanned by the customer at the self-checkout device.

[0030]

[0035] In an embodiment where the feature data 325 is sent directly to the fraud detection component 340, the fraud detection component 340 processes the checkout data to identify the quantity of items scanned by the customer and can then compare the scanned quantity with the quantity identified from the 3D model. Such a comparison can enable the fraud detection component 340 to detect a mismatch between the items scanned at self-checkout and the items physically present in the container. Upon detecting a mismatch (e.g., a quantity mismatch), the fraud detection component 340 can issue a warning 350 and / or send the warning to the store clerk. In some embodiments, the warning can notify the store clerk of potential problems such as some items in the container not being scanned or there being an error in the self-checkout device. In some embodiments, the warning can suggest immediate actions to the store clerk such as rescanning the items that were accidentally overlooked or not scanned, manually verifying the contents of the container, or resetting the self-checkout device to address any technical issues that may have caused the error.

[0031]

[0036] In an embodiment where the feature data 325 is sent to the object recognition component 330, the object recognition component 330 can interpret the feature data 325 to identify each individual item (e.g., a water bottle) or classify each item into a broader category (e.g., bottled beverages). The identification process can include first training a machine learning model (e.g., a convolutional neural network (CNN)) to learn features from known items, and then comparing the extracted features such as the size, shape, and / or material of each item in the container with the learned features. In some embodiments, this comparison can lead to generating a corresponding matching score for each item in the container. In some embodiments, the matching score can represent how closely the features of the item in the container match the features of the known items in the database. In some embodiments, the object recognition component 330 can label the item in the container with the category or item identification information label having the highest matching score. For example, if the feature data of the item in the container closely matches the learned features of a water bottle in the database, the object recognition component 330 can identify and label it as a water bottle. Under such a configuration, the object recognition component 330 can generate a list 335 (also referred to as an identification information list in some embodiments) that identifies each item such as one water bottle or two packages of chips. If the feature data of the item in the container closely matches the learned features of a broader category such as bottled beverages, the component 330 can classify the item under the broader category. Under such a configuration, the object recognition component 330 can generate a list 335 that details the quantity and category of the items, such as "one bottled beverage", "two snack foods", or "four fresh foods".

[0032]

[0037] In the illustrated example, the identification information list 335 is then provided to a fraud detection component 340 that compares the list 335 to the checkout data 345 to detect discrepancies that may indicate the occurrence of fraud. In some embodiments, the fraud detection component 340 may process the checkout data to extract a transaction list. In some embodiments, the transaction list may refer to a record of items scanned by a customer during self-checkout by that customer. In some embodiments, the transaction list may include detailed information such as the type, quantity, and / or price of each scanned item.

[0033]

[0038] Subsequent to the extraction of the transaction list, in some embodiments, the fraud detection component 340 may compare the transaction list to the identification information list 335 (generated by the object recognition component 330). This comparison is designed to detect discrepancies between the items identified within the container and the items actually scanned by the customer. For example, if the identification information list 335 indicates three snack foods but the transaction list only includes two, this mismatch may indicate an item missed by the customer or an item scanned incorrectly (e.g., one item scanned multiple times by mistake), or a potential error in the self-checkout process. Upon identifying such a discrepancy, the fraud detection component 340 may send a warning 350 to the store clerk requesting further action, such as manual verification or resetting the self-checkout device. In some embodiments, the detected discrepancy may not necessarily indicate fraud. Rather, the discrepancy may be the result of a simple user error, such as forgetting to scan an item or scanning one item multiple times. In such a configuration, the fraud detection component 340 may prompt the self-checkout device to send a message notifying the customer of the discrepancy and suggesting a rescan. By doing so, the fraud detection component 340 may effectively prevent losses for both the customer and the business owner to ensure that the transaction is correctly billed, protect the customer from unintentional overcharging, and protect the business owner from undercharging.

[0034]

[0039] Figure 4 shows an exemplary method 400 for issuing a warning in response to potential checkout fraud based on collected sensor data and checkout data according to some embodiments of the present disclosure. In some embodiments, the exemplary method 400 may be implemented by one or more computing systems such as server 135 shown in FIG. 1 and / or computing device 600 shown in FIG. 6.

[0035]

[0040] Method 400 begins at block 405 where a computing system (e.g., central server 135 of FIG. 1) collects data from one or more acoustic wave sensors (e.g., 115 of FIG. 1). In some embodiments, the acoustic wave sensors can be installed in a scan area (e.g., 145 of FIG. 1) within or near a self-checkout queue at a business site (e.g., a retail store). Setting the scan area ensures that waiting customers and their containers (e.g., 125 of FIG. 1) (e.g., carts, baskets, bags) reach the self-checkout device (e.g., 110 of FIG. 1) after passing through the scan zone. To achieve comprehensive coverage, in some embodiments, the acoustic wave sensors (e.g., 115 of FIG. 1) can be placed at various locations around the scan zone to capture the contents of the containers from various perspectives. Sensor data is collected as the customer pushes their container through the scan area. The sensor (e.g., 115 of FIG. 1) emits acoustic waves (e.g., 220a and 220c of FIG. 2) towards the container and captures the acoustic waves (e.g., 220b and 220d of FIG. 2) reflected by the container or its contents. The sensor data can include raw wave data that includes both the emitted and reflected waves. In some embodiments, such as when the sensor has built-in computing capabilities, the sensor can preprocess the wave data before transmitting it to the computing system. For example, the sensor can filter the wave data to remove noise or irrelevant frequencies, or analyze the wave data to identify relevant parameters such as time of flight (e.g., the time it takes for the wave to travel to and return from the container), amplitude (or intensity) of the received wave, and frequency changes. In some embodiments, these parameters can be used to generate a visual representation of the contents of the container. In some embodiments, the computing system can collect sensor data from various acoustic wave sensors (e.g., sensors installed at various positions around the scan area) and aggregate the data into a single dataset.

[0036]

[0041] In block 410, the computing system analyzes the received sensor data and generates a visual representation (e.g., a 3D model) of the contents of the container. In some embodiments, as discussed above, the sensor data may include raw wave data and processed parameters. In some embodiments, the visual representation may include a 3D model showing the contours and spatial arrangement of the items within the container. In some embodiments, the generation of the model may be based on integrating sensor data collected from various acoustic wave sensors (e.g., those installed at various positions around the scan area). For example, in some embodiments, the contours or shapes of the items within the container may be inferred by analyzing the amplitude and frequency data captured by these sensors. Each sensor contributes to a multi-dimensional view of the items within the container by capturing waves from its unique angle. In some embodiments, the spatial arrangement of the items within the container may be determined by calculating the distances from the sensors to various points within the container. Such calculations may be achieved by analyzing time-of-flight data that measures the time it takes for an acoustic wave to propagate to the container, be reflected by the items, and return to the sensors.

[0037]

[0042] In block 415, the computing system analyzes the model to identify features or characteristics of the items within the container. This process includes extracting overall or individual item features. For example, in some embodiments, the computing system may evaluate the total number of items within the container. In some embodiments, the computing system can examine each item to determine its shape, size, or material composition.

[0038]

[0043] At block 420, the computing system utilizes the extracted feature data to identify each item within the container. In some embodiments, the object recognition process may include training an ML model (e.g., a CNN) with a vast dataset of known items, in which case the model learns to recognize the features of each known item. Once the model training is complete, the computing system may use the trained model to analyze the features extracted from the items within the container. Each item within the container can be compared to the features learned by the ML model. The system can evaluate how closely the features of the items within the container match the features of the known items in the dataset. In some embodiments, this comparison results in a matching score for each item, which indicates the likelihood that the item is a known / target product (e.g., a water bottle) or belongs to a broader category (e.g., bottled beverages). In some embodiments, the computing system can label each item within the container with the category or item identification information label corresponding to the highest matching score from the ML analysis. Following object recognition, the system can then compile that information into an identification information list that details the quantity and / or category of the items identified within the container. For example, in some embodiments, the list can include item identification information and their respective quantities, such as two water bottles or two packs of chips. In some embodiments, the list can include general categories and their respective quantities, such as "one bottled beverage," "two snack foods," or "four fresh foods."

[0039]

[0044] At block 425, the computing system analyzes the checkout data provided by the self-checkout system. In some embodiments, the checkout data can include a list of items (along with their respective quantities) that the customer scanned themselves during their self-checkout process.

[0040]

[0045] At block 430, the computing system compares the items identified from the sensor data with the items identified from the checkout data. This comparison is designed to identify whether the quantity or type of items detected within the container (by the acoustic sensor) matches the items scanned by the customer at the self-checkout device. If the computing system identifies a discrepancy (e.g., an item present within the container but not reflected in the checkout data), method 400 proceeds to block 435 where the computing system issues a warning. In some embodiments, this warning may be sent to a store employee to notify the employee of a potential problem and / or may require immediate action such as manual verification to resolve the discrepancy. In some embodiments, as described above, the discrepancy may be caused by user error such as forgetting to scan an item or scanning an item multiple times. In such a configuration, when a discrepancy is detected, the computing system instructs the self-checkout device to display a message to notify the customer of the discrepancy and to suggest a rescan. If the comparison shows no discrepancy, indicating that the customer's checkout process was accurate and compliant, method 400 returns to block 405 and the computing system resumes the operation of collecting sensor data for the next container. This loop ensures continuous monitoring and fraud detection for each transaction at the self-checkout station.

[0041]

[0046] In some embodiments, object recognition shown at block 420 may be optional. For example, the computing system may utilize general item characteristics (e.g., the total quantity of items within a container) for fraud detection. Under such a configuration, method 500 proceeds directly to block 425, and the computing system analyzes checkout data to determine the quantity of items actually scanned by a customer at a self-checkout device. Following this determination, the computing system compares the two quantities (e.g., the quantity determined from the checkout data and the quantity determined from the sensor data). If a discrepancy is detected, such as the quantity from the sensor data being greater than the quantity from the checkout data, this may indicate that there are items within the container that have not been scanned or have been scanned inappropriately. Method 500 then proceeds to block 435, where the computing system requests manual verification and alerts a store employee. If no discrepancy is shown, such as the quantity from the sensor data matching the quantity from the checkout data, this may indicate that all items within the current container have been scanned appropriately. Method 500 returns to block 405, and the computing system continues to collect sensor data for the next container.

[0042]

[0047] FIG. 5 is a flowchart showing an exemplary method 500 for fraud detection according to some embodiments of the present disclosure.

[0043]

[0048] At block 505, a computing system (e.g., central server 135 of FIG. 1) receives sensor data (e.g., 305 of FIG. 3) from one or more acoustic wave sensors (e.g., 115 of FIG. 1) that transmit acoustic waves (e.g., 220a of FIG. 2) towards a set of items within a container and receive reflected acoustic waves (e.g., 220b of FIG. 2) from the set of items.

[0044]

[0049] At block 510, the computing system generates a model (e.g., 315 in FIG. 3) of a set of items within the container based on sensor data (e.g., 305 in FIG. 3). In some embodiments, the model may comprise a three-dimensional model of the set of items within the container.

[0045]

[0050] At block 515, the computing system identifies one or more characteristics (e.g., 325 in FIG. 3) of the set of items by analyzing the model. In some embodiments, the one or more characteristics may comprise at least one of the number of items in the set of items within the container, the shape of one of the items in the set of items, the size of one of the items in the set of items, or the material composition of one of the items in the set of items.

[0046]

[0051] At block 520, the computing system retrieves checkout data (e.g., 345 in FIG. 3) from one or more checkout devices (e.g., 110 in FIG. 1). In some embodiments, the checkout data may comprise a transaction list of items scanned by a user at the one or more checkout devices.

[0047]

[0052] At block 525, the computing system compares one or more characteristics (e.g., 325 in FIG. 3) of the set of items identified from the model with the checkout data (e.g., 345 in FIG. 3).

[0048]

[0053] At block 530, the computing system issues a warning (e.g., 350 in FIG. 3) when a discrepancy is detected between the one or more characteristics and the checkout data. In some embodiments, the discrepancy may comprise a difference in the number, shape, or size of the items in the set of items.

[0049]

[0054] In some embodiments, the computing system may extract depth information for each item in a set of items within a container, based on respective time - of - flight, where each time - of - flight is measured from the time an acoustic wave is transmitted towards each item to the time the corresponding reflected acoustic wave is received. In some embodiments, the depth information for each item may comprise respective distances between each item and one or more acoustic sensors.

[0050]

[0055] In some embodiments, the computing system may generate a list of item identification information from a model by comparing the identified one or more features with acoustic signatures corresponding to known items, and may compare this list of identification information with a transaction list to detect discrepancies.

[0051]

[0056] FIG. 6 shows an exemplary computing device 600 configured to implement various aspects of the present disclosure, according to some embodiments of the present disclosure. Although shown as a physical device, in some embodiments, the computing device 600 may be implemented using virtual device(s) and / or across several devices (e.g., in a cloud environment). The computing device 600 may be embodied as any computing device, such as the central server 135 shown in FIG. 1, the model generation component 310, the feature extraction component 320, the object recognition component 330, and the fraud detection component 340 shown in FIG. 3.

[0052]

[0057] As shown in the figure, computing device 600 includes a CPU 605, a memory 610, a storage 615, one or more network interfaces 625, and one or more I / O interfaces 620. In the illustrated embodiment, the CPU 605 fetches and executes programming instructions stored in the memory 610, and stores and retrieves application data present in the storage 615. The CPU 605 generally represents a single CPU and / or GPU, multiple CPUs and / or GPUs, a single CPU and / or GPU having multiple processing cores, etc. The memory 610 is generally considered to represent random access memory. The storage 615 can be any combination of a disk drive, a flash-based storage device, etc., and can include fixed and / or removable storage devices such as a fixed disk drive, a removable memory card, a cache, an optical storage, a network-attached storage (NAS), or a storage area network (SAN).

[0053]

[0058] In some embodiments, I / O devices 635 (such as a keyboard, a monitor, etc.) are connected via the I / O interface(s) 620. Further, via the network interface 625, the computing device 600 can be communicatively coupled to one or more other devices and components (e.g., via a network (s) that can include the Internet, a local network, etc.). As shown, the CPU 605, the memory 610, the storage 615, the network interface(s) 625, and the I / O interface(s) 620 are communicatively coupled by one or more buses 630.

[0054]

[0059] In the illustrated embodiment, the memory 610 includes a model generation component 650, a feature extraction component 655, an object recognition element 660, and a fraud detection component 665. Although illustrated as separate components for conceptual clarity, in some embodiments, the operations of the illustrated components (and others not shown) may be combined or distributed among any number of components. Further, although illustrated as software present in the memory 610, in some embodiments, the operations of the illustrated components (and others not shown) may be implemented using hardware, software, or a combination of hardware and software.

[0055]

[0060] In the illustrated embodiment, the model generation component 650 (which may correspond to the model generation component 310 of FIG. 3) receives sensor data from one or more acoustic wave sensors (e.g., the sensor 115 installed in the scan area 145 shown in FIG. 1), and processes the data to construct a model (e.g., a 3D model) indicative of the contents of the container (e.g., the cart 125-2 passing through the scan area 145 shown in FIG. 1). In some embodiments, the sensor data (e.g., 305 of FIG. 3) may include information regarding the emitted acoustic wave and the received acoustic wave. In some embodiments, such as when the sensor has a built-in computing function, the sensor data may include wave data along with parameters identified from the wave data, such as time of flight, amplitude, and frequency changes. In some embodiments, the model (e.g., 315 of FIG. 3) generated by the model generation component 650 may indicate the contour of the item and the spatial arrangement of the items within the container. In some embodiments, the model may be generated by analyzing the sensor data in various ways. For example, in some embodiments, by analyzing the time of flight data, the model generation component 650 may determine the distance or position of each item within the container. In some embodiments, by examining the frequency change and / or amplitude of the reflected wave, the component 650 may infer the shape, size, or material properties of the item.

[0056]

[0061] In the illustrated embodiment, the feature extraction component 655 (which may correspond to the feature extraction component 320 of FIG. 3) is configured to analyze a model (e.g., 315 of FIG. 3) and / or sensor data (e.g., 305 of FIG. 3) to extract features or characteristics related to the items within the container (e.g., 325 of FIG. 3). For example, in some embodiments, the feature extraction component 655 may identify the dimensions or shape of each item and / or infer the material of each item based on the sensor data. In some embodiments, the feature extraction component 655 may evaluate the total quantity of items within the container. By analyzing the depth information and / or the spatial relationships between the items indicated in the model, the component 655 can detect whether the items are stacked or hidden, and thus ensure that all items within the container are properly identified and accounted for. In some embodiments, the extracted features (e.g., 325 of FIG. 3) may then be provided to the fraud detection component 665 and / or the object recognition component 660 for further processing and analysis.

[0057]

[0062] In the illustrated embodiment, the object recognition component 660 (which may correspond to the object recognition component 330 of FIG. 3) is designed to identify each item or classify each item into a broader class based on the extracted features. In some embodiments, the object recognition component 660 may train an ML model (e.g., a CNN) to learn features from known items. During the training phase, the model recognizes and interprets the characteristics (e.g., shape, size, material) of various known items. Once the training is complete, the object recognition component 660 can then compare the extracted features from the items in the container to the learned features. This comparison can result in the calculation of a matching score that quantifies the similarity between the items in the container and the known items. The higher the score, the more likely that item matches the known characteristics. Based on the score, the object recognition component 660 can assign to each item the item identification information or the label of a broader category corresponding to the highest matching score. In some embodiments, the output of the object recognition component 660 may include a list of identification information that details each item or each category identified in the container along with their respective quantities.

[0058]

[0063] In the illustrated embodiment, the fraud detection component 665 (which may correspond to the fraud detection component 340 of FIG. 3) is configured to analyze checkout data (provided by a self-checkout system at a business site) and to compare that checkout data with extracted features of the items (such as the quantity or size of the items) or with a list of identification information generated by the object recognition component 660. In some embodiments, the checkout data may include a transaction list detailing the items actually scanned by a customer at a self-checkout device. The comparison performed by the fraud detection component 665 is intended to detect a discrepancy between the items identified within a container by an acoustic wave sensor and the items actually scanned by the customer. For example, in some embodiments, features identified from sensor data, such as the quantity or size of an item, may be compared with the checkout data. In some embodiments, such as when the list of identification information is generated by the object recognition component 660, the fraud detection component 665 may perform verification on a per-item or per-class basis by comparing the list of identification information with the transaction list from the checkout data. If this comparison indicates a discrepancy, such as a quantity or size mismatch, or a mismatch between the list of identification information and the transaction list, it may indicate a potential problem. These potential problems may include items missed by the customer or items scanned incorrectly (e.g., one item scanned multiple times by mistake), or potential errors in the self-checkout system. When such a discrepancy is detected, the fraud detection component 665 may alert a store employee, notify of the potential problem, and request measures such as manual verification.

[0059]

[0064] In the illustrated example, storage 615 may include past checkout data 675, past sensor data 680, and past fraud detection results 685. In some embodiments, the past checkout data 675 may include records of transactions performed by a self-checkout device (e.g., 110 of FIG. 1). In some embodiments, the past sensor data 675 may include raw wave data such as emitted acoustic waves and received acoustic waves, as well as processed data such as time of flight, amplitude of reflected waves, and frequency changes. In some embodiments, the past fraud detection results 680 may include a log of discrepancies and potential fraudulent acts detected by the fraud detection component 665. In some embodiments, each of the past fraud detection records 680 may detail the nature of the discrepancy, the items involved, the time of occurrence, and any measures taken in response. In some embodiments, the past checkout data 675, the past sensor data 680, and the corresponding past fraud detection results 680 may be analyzed to understand common patterns of checkout discrepancies and / or to improve the fraud detection system for better performance. In some embodiments, the foregoing data may be stored in a remote database (e.g., 140 of FIG. 1) connected to the computing device 600 via a network (e.g., 130 of FIG. 1).

[0060]

[0065] The description of various embodiments of the present disclosure has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been chosen to best explain the principles of the embodiments, the practical application, or technical improvements found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0061]

[0066] The following refers to the embodiments presented in the present disclosure. However, the scope of the present disclosure is not limited to the described embodiments. Instead, any combination of the following features and elements is contemplated for implementing and practicing the contemplated embodiments, regardless of whether they relate to various embodiments. Furthermore, the embodiments disclosed herein may achieve other possible solutions or advantages over the prior art, but whether one advantage is achieved by a given embodiment does not limit the scope of the present disclosure. Therefore, the following aspects, features, embodiments, and advantages are merely exemplary and are not considered to be elements or limitations of the claims, unless explicitly recited in the following claims (one or more).

[0062]

[0067] Aspects of the present disclosure can take the form of embodiments that are entirely hardware, embodiments that are entirely software (including firmware, resident software, microcode, etc.), or embodiments that combine software aspects and hardware aspects, all of which may generally be collectively referred to herein as "circuits", "modules", or "systems".

[0063]

[0068] The present disclosure can be a system, method, and / or computer program product. The computer program product can include a computer-readable storage medium (one or more) having computer-readable program instructions for causing a processor to execute aspects of the present disclosure.

[0064]

[0069] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of further examples of computer-readable storage media includes portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy (registered trademark) disks, punch cards, or mechanical encoding devices such as raised structures in the form of grooves in which instructions are recorded, and any suitable combination of the foregoing. A computer-readable storage medium, as used herein, should not be considered to be a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through an electrical wire.

[0065]

[0070] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing devices / processing devices or to an external computer or external storage device via a network such as, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface of each computing device / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within each computing device / processing device.

[0066]

[0071] Computer-readable program instructions for carrying out the operations of this disclosure may be source code or object code written in any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit for implementing aspects of this disclosure.

[0067]

[0072] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0068]

[0073] These computer-readable program instructions are provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine that causes the instructions executed by the processor of the computer or other programmable data processing apparatus to implement the functions / operations specified within one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other device to function in such a way that a product including the instructions stored therein implements the functions / operations specified within one or more blocks of a flowchart and / or block diagram.

[0069]

[0074] These computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer implemented process such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / operations specified within one or more blocks of a flowchart and / or block diagram.

[0070]

[0075] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible embodiments of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing the specified logical function(s). In some alternative embodiments, the functions shown within the block may be performed out of the order shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or may sometimes be executed in the reverse order depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0071]

[0076] Embodiments of the present disclosure may be provided to an end user through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as services over a network. More formally, cloud computing is defined as a computing function that enables convenient on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction by performing an abstraction between the computing resources and their underlying technical architecture (e.g., servers, storage, networks). Thus, cloud computing enables a user to access virtual computing resources of the "cloud" (e.g., storage, data, applications, and even complete virtualized computing systems) without concern for the underlying physical systems (or the location of these systems) used to provide the computing resources.

[0072]

[0077] Typically, cloud computing resources are provided to a user on a pay-per-use basis, where the user is billed only for the computing resources actually used (e.g., the amount of storage space consumed by the user or the number of virtualized systems instantiated by the user). The user can access any of the resources present in the cloud at any time and from anywhere on the Internet. In the context of the present disclosure, a user may access an application available in the cloud (e.g., a fraud detection application) or related data. For example, a fraud detection application may perform data processing and object recognition through the cloud computing infrastructure and store relevant results in a cloud storage location. By doing so, the user can access this information from any computing system connected to a network (e.g., the Internet) connected to the cloud.

[0073]

[0078] The above is directed to embodiments of the present disclosure, but other further embodiments of the present disclosure may be devised without departing from its basic scope, which is determined by the following claims.

Claims

1. Receiving sensor data from one or more acoustic wave sensors that transmit acoustic waves toward a set of items within a container and receive reflected acoustic waves from the set of items; Generating a model of the set of items within the container based on the sensor data; Identifying one or more characteristics of the set of items by analyzing the model; Retrieving checkout data from one or more checkout devices; Comparing the one or more characteristics of the set of items identified from the model with the checkout data; Issuing a warning when a discrepancy is detected between the one or more characteristics and the checkout data; A method comprising the above steps.

2. The method of claim 1, wherein the one or more characteristics comprise at least one of (i) the number of items in the set of items within the container, (ii) the shape of one item in the set of items, (iii) the size of one item in the set of items, or (iv) the material composition of one item in the set of items.

3. The method of claim 1, wherein the discrepancy comprises a difference in the number, shape, or size of the items in the set of items.

4. The method of claim 1, further comprising extracting depth information based on a time of flight, which is the time from when the acoustic wave is transmitted toward each item in the set of items within the container until the corresponding reflected acoustic wave is received, for each item in the set of items within the container.

5. The method of claim 4, wherein the depth information for each item comprises the respective distance between each item and the one or more acoustic wave sensors.

6. The method of claim 1, wherein the model comprises a three-dimensional model of the set of items within the container.

7. The method of claim 1, wherein the checkout data comprises a transaction list of items scanned by a user at the one or more checkout devices.

8. Generating a list of item identification information from the model by comparing the identified one or more characteristics with acoustic signatures corresponding to known items; The method of claim 7, further comprising comparing the list of identification information with the transaction list to detect the discrepancy.

9. One or more processors; One or more memories that store a program that performs an operation when executed on any combination of the one or more processors; A system comprising: the operation being Receiving sensor data from one or more acoustic wave sensors that transmit acoustic waves towards a set of items in a container and receive reflected acoustic waves from the set of items; Generating a model of the set of items in the container based on the sensor data; Identifying one or more features of the set of items in the container by analyzing the model; Retrieving checkout data from one or more checkout devices; Comparing the one or more features of the set of items identified from the model with the checkout data; Issuing a warning when a discrepancy is detected between the one or more features and the checkout data; A system comprising.

10. The system according to claim 9, wherein the one or more features comprise at least one of (i) the number of items in the set of items in the container, (ii) the shape of one item in the set of items, (iii) the size of one item in the set of items, or (iv) the material composition of one item in the set of items.

11. The system according to claim 9, wherein the program, when executed on any combination of the one or more processors, further performs the operation of extracting depth information based on respective time-of-flight times, which are the times from when the acoustic waves are transmitted towards each respective item in the set of items in the container until the corresponding reflected acoustic waves are received, for each respective item in the set of items in the container.

12. The system according to claim 11, wherein the depth information for each respective item comprises the respective distance between each respective item and the one or more acoustic wave sensors.

13. The system according to claim 9, wherein the model comprises a three-dimensional model of the set of items in the container.

14. The system according to claim 9, wherein the checkout data comprises a transaction list of items scanned by a user at the one or more checkout devices.

15. The program performs an operation when executed on any combination of the one or more processors, the operation being Generating a list of item identification information from the model by comparing the identified one or more features with acoustic signatures corresponding to known items; Comparing the list of identification information with the transaction list to detect the mismatch; The system according to claim 14, further comprising.

16. When executed by the operation of a computer system, Receiving sensor data from one or more acoustic wave sensors that transmit acoustic waves towards a set of items in a container and receive reflected acoustic waves from the set of items; Generating a model of the set of items in the container based on the sensor data; Identifying one or more features of the set of items by analyzing the model; Retrieving checkout data from one or more checkout devices; Comparing the one or more features of the set of items identified from the model with the checkout data; Issuing a warning when a mismatch is detected between the one or more features and the checkout data; One or more non-transitory computer-readable media comprising computer program code for performing operations in any combination.

17. The one or more non-transitory computer-readable media according to claim 16, wherein the one or more features comprise at least one of (i) the number of items in the set of items in the container, (ii) the shape of one item in the set of items, (iii) the size of one item in the set of items, or (iv) the material composition of one item in the set of items.

18. The one or more non-transitory computer-readable media according to claim 16, wherein when executed by the operation of a computer system, the computer program code for performing the operations further comprises extracting depth information based on the respective time of flight of each acoustic wave from the time when the acoustic wave is transmitted towards each item in the set of items in the container until the corresponding reflected acoustic wave is received for each item in the set of items.

19. The one or more non-transitory computer-readable media according to claim 18, wherein the depth information of each item comprises the respective distance between each item and the one or more acoustic wave sensors.

20. The one or more non-transitory computer-readable media of claim 16, wherein the model comprises a three-dimensional model of the set of items within the container.

Images