System and method for verifying a product placed in a shopping cart and / or shopping basket

Abstract

The aspects described herein relate to a method (600) for automatically constructing a weight and / or image database for verifying a product, involving: obtaining (610) a first measured weight and / or a first captured image of the product from a stationary cash register; obtaining (620) a second measured weight and / or a second captured image of the product from a shopping cart and / or a shopping basket; and adding (630) the first and the second measured weight and / or the first and the second captured image of the product to the weight and / or image database.

Description

[0001] January 12, 2026 KBST GmbH K177738WO ANE / Snj

[0002] SYSTEM AND METHOD FOR VERIFYING A PRODUCT PLACED IN A SHOPPING CART AND / OR SHOPPING BASKET

[0003] 1. Technical field

[0004] One aspect concerns a procedure, a computer program and a system for verifying a product placed in a shopping cart and / or shopping basket.

[0005] A second aspect concerns a method, a computer program and a system for the automated creation of an image database and / or the automated training of a model to verify a product placed in a shopping cart and / or shopping basket.

[0006] 2. Background

[0007] Shopping in a self-service store traditionally involves paying for goods at a checkout upon completion of the purchase. The items selected by the customer are identified at the checkout, allowing the customer to pay accordingly. This identification is often carried out by staff, such as a cashier, using a barcode scanner and / or scales. Alternatively, or additionally, self-checkout lanes are used where there is no cashier; instead, the customer scans and pays for the items themselves. Typically, a staff member is stationed near such a self-checkout lane to assist with any problems and to monitor the process.

[0008] However, in both cases, staff must be assigned to actively identify products and / or monitor this process. Furthermore, traditional checkouts, which typically include a conveyor belt, as well as self-checkouts, which require separate payment and scanning terminals, are space-consuming. In particular, there is only a limited number of checkouts and / or terminals in a store, generally fewer than the number of customers present. Therefore, queues form at the checkout area, especially during peak hours, as multiple customers are assigned to one checkout and there isn't an individual checkout for each customer. This led to the development of digital shopping carts, where the products intended for purchase by the customer are identified as soon as they are placed in the cart. This identification is intended to occur without the assistance of staff.However, this necessitates the use of certain control mechanisms. These control mechanisms are designed to ensure that all products placed in the shopping cart are registered and match the products for which the customer ultimately pays. The control mechanisms used so far can be divided into two classes.

[0009] The first generation of digital shopping carts uses a checkweigher to ensure that the customer correctly records all items placed in the cart. This checkweigher weighs the product and compares the measured weight to a weight stored in a database for that specific scanned product. However, this is problematic, particularly in the food sector, especially fast-moving consumer goods (FMCG) (products with high turnover, i.e., products with a short average storage time in a warehouse and / or on a retail shelf; typical examples include food and beverages, personal care products, cleaning supplies, and especially daily newspapers), and in DIY and furniture stores. In these sectors, many products have the same, or at least very similar, weight but differ significantly in price.The checkweigher cannot differentiate between such products, or can only do so inadequately, and consequently, it cannot be guaranteed that the product placed in the shopping scales matches the item being recorded. There is also the problem that the weight of products is often subject to variations, for example, due to production processes, so that a precise weight cannot be recorded.

[0010] The second class of digital shopping carts uses optical image recognition via camera to ensure that the customer correctly identifies all items placed in the cart. However, this proves difficult in practice, as typical supermarkets have to distinguish between tens of thousands of products, and these products can be viewed from all possible angles, from all sides, and partially obscured when placed in shopping carts.

[0011] Another problem with optical image recognition is that the product placed in the shopping cart must be visually accessible. If the product is not visible to the camera, it cannot be optically detected and identified. This visual inaccessibility of products in the shopping cart can be caused by the customer either unintentionally or intentionally. Firstly, the product might be positioned in such a way that it is obscured by another product already in the cart. Secondly, the customer might deliberately hide a product inside the packaging of another product. However, since the camera only checks the optical properties of the product itself, it cannot register the difference.

[0012] Another disadvantage of digital shopping carts, which are based either on a control scale or optical image recognition, is that despite the control mechanisms, staff are still required at some point in the shopping process. If the customer has a problem scanning the product or if the product is scanned incorrectly, staff must assist and support the customer. However, qualified personnel are difficult to find and expensive in times of skilled labor shortages.

[0013] KR 102323796 Bi concerns a shopping cart that recognizes products using artificial intelligence. The shopping cart has a camera for taking pictures of products placed in it. These images are transmitted to a server, which identifies the product using artificial intelligence.

[0014] EP 3262562 Bi relates to a system and a method for identifying products in a shopping cart. This identification is based on optical image recognition software using a camera attached to the shopping cart and a comparison with a database containing product-specific information.

[0015] In light of this technical background, there is a need for improved methods to verify a product placed in a shopping cart and / or shopping basket.

[0016] 3. Summary of the invention

[0017] This goal is achieved in a first aspect by a procedure for verifying a product placed in a shopping cart and / or shopping basket, which includes the following steps: receiving at least one image of the placed product; selecting a trained model based on a scanned product; and determining a probability that the at least one image of the placed product shows the scanned product, based on the trained model.

[0018] The at least one received image of the placed product is an image of the same product being placed in the shopping cart and / or shopping basket and is preferably generated in connection with a product placement process. This placement process can include the product lying in the shopping cart, the act of placing it in a shopping cart, as well as picking out a product, preferably from a shelf and / or from a display device. The at least one image can comprise several or a multitude of images and / or include images from a video sequence. For example, the at least one image and / or the video sequence can be generated by one or more cameras in the shopping cart and / or shopping basket. The scanned product for which a trained model is selected can differ from the placed product, e.g.,This occurs when the product placed in the shopping cart or basket is not the one that was previously scanned. The trained model is based on optical or optically accessible properties of the scanned product. Optically accessible properties include length, width, height, color, and geometry. The probability that the scanned product does not differ from the placed product is quantified. Determining this probability, based on the trained model, indicates the likelihood that at least one image of the placed product shows the scanned product. Verification of the placed product is based on this determined probability. Generally, the scanned product could be a product that will be billed to the customer at the end of the shopping process.Verification ensures that the product placed actually corresponds to the scanned (and invoiced) product (and not, for example, a significantly more expensive product, for which the cheaper price of the scanned product would then be incorrectly invoiced).

[0019] The following examples specifically refer to a shopping cart. However, all aspects can also apply to a shopping basket, even if this is not always explicitly stated. The term "shopping cart" is used to encompass all carts typically used to transport goods from a sales area to a checkout, including platform carts, stake carts, pipe trolleys, magazine carts, roll containers, storage carts, etc.

[0020] The trained model for the scanned product can be based on an image database.

[0021] In particular, selecting the trained model can involve calling the trained model from memory. The selection can be made from a plurality of different trained models. For example, the trained model may have been trained using a subset of the images in the image database. This allows determining the probability that at least one image of the inserted product shows the scanned product by applying the trained model to that at least one image.

[0022] In particular, the image database can be specific to the scanned product. A specific image database contains only images for a specific product. The specific image database can be a separate database for each product, distinct from the image databases of other products. Similarly, the trained model can be specifically trained for the scanned product. The specifically trained model can therefore be trained, for example, to recognize precisely the scanned product. For this purpose, in addition to the image database for the scanned product, images of other products can also be used as "negative examples" for training. With a model specifically trained for the scanned product, for example, only a probability can be calculated indicating that at least one image of the inserted product shows the scanned product.The hypothesis being tested is whether the scanned product matches the product placed in the tray. Hypotheses regarding products other than the scanned product may not be possible with a model specifically trained for the scanned product.

[0023] In some examples, a product can be scanned, for instance, with a barcode scanner on a shopping cart. Subsequently, a (possibly different) product can be placed in a shopping cart, and at least one image of the placed product can be taken, for example, by one or more cameras attached to the shopping cart. Based on the scanned product and / or information about the scanned product, a trained model can then be selected, for example, a model specifically trained for the scanned product. Using this model and the at least one image, a probability (e.g., in %) can then be determined as to whether the at least one image shows the scanned product (it is emphasized that responses such as "yes" / "no" and possibly "unclear" can also be interpreted as probabilities without having to specify a concrete percentage).

[0024] Some steps of the procedure can be executed by a computer belonging to the shopping cart (e.g., a tablet mounted on the shopping cart). Alternatively, all steps of the procedure can be executed by a computer belonging to the shopping cart. For example, the trained model and any other selected trained models can be stored on the shopping cart's memory and / or its computer. Alternatively or additionally, some steps can be executed by a server and / or in the cloud. For example, all steps of the procedure can be executed by the server and / or the cloud. In other embodiments, a first part of the procedure can be executed by the computer belonging to the shopping cart, and a second part of the procedure can be executed by the server and / or the cloud. The first and second parts can be disjoint.Alternatively, certain steps of the process can be performed by the computer belonging to the shopping cart as well as by the server and / or the cloud.

[0025] For example, the computer belonging to the shopping cart can receive at least one image of the product placed inside, preferably from one or more cameras belonging to the shopping cart. Then, as a first alternative, the shopping cart and / or its computer can send at least one image and information about the scanned product to the server and / or the cloud. For this purpose, a wireless communication device belonging to the shopping cart and / or its computer can be used (such as a transmitter / receiver with Wi-Fi, Bluetooth, 3G, 4G, 5G, etc.). The server and / or the cloud then selects a trained model based on the scanned product and / or the information about the scanned product. Based on the trained model, the server and / or the cloud determines a probability that the at least one image of the product placed inside depicts the scanned product.The calculated probability can be sent back to the shopping cart. The server and / or cloud can belong to a market associated with the shopping cart and / or to a corresponding international, national, or regional headquarters of a department store group. Alternatively, the shopping cart and / or its computer can send (only) information about the scanned product to the server and / or cloud. The server and / or cloud then selects a trained model based on the scanned product or the information about the scanned product. The server and / or cloud can then, for example, send the trained model to the shopping cart and / or its computer, which then determines the probability that at least one image of the product in the cart shows the scanned product, based on the trained model.In this case too, the server and / or the cloud may belong to a marketplace associated with the shopping cart and / or to a corresponding international, national, or regional headquarters of a department store group.

[0026] The computers, servers, and / or clouds described below can include one or more processors and one or more storage devices. Each processor can include one or more processor cores, and each processor can include one or more logic circuits for processing data and / or information. For example, each processor can include an arithmetic logic unit (ALU), a control unit, and a variety of registers. Each processor can include cache memory. Each processor can comprise a system-on-a-chip (SoC), which includes a variety of processor cores, random-accessible read / write memory (RAM / DRAM), graphics processing units (GPUs), one or more controllers, and one or more communication modules / interfaces. Each processor can include a variety of transistors. In general, a computer and / or computer system can be configured to receive and / or forward data.This data can be read from machine-readable storage media such as hard drives, magnetic disks / floppy disks, solid-state drives (SSDs), magneto-optical disks (MODs), or optical disks. A computer program can be written in any programming language, in particular a compiled language and / or an interpreted language and / or a scripting language. Examples of possible languages ​​include C, C++, Fortran, Python, and Perl.

[0027] Using a specifically trained model maximizes the quality of hypothesis testing, minimizes the probability of erroneous verification, and thus optimizes the security of the purchasing process. In particular, a specifically trained model can deliver a unique and accurate output with a very high probability. Generic optical image recognition models have the disadvantage that they often produce unusable results; that is, the output of the image recognition model is not conclusive. This is primarily due to the use of a global image recognition model trained on all products offered in a market. Since the product range of a market typically comprises 40,000 to 60,000 items, this is not sufficient.With 000 products, the image recognition model for a placed product provides a list of match probabilities for each product in the assortment. This would require extremely complex training for a single model to reliably recognize all of the vast number of products. In contrast, models specifically trained for individual products are significantly faster and more reliable. Furthermore, the runtime of the specifically trained model is minimized because the model is smaller and / or less complex. This leads to faster verification of the placed product and thus a smoother and more pleasant shopping experience for the customer. Additionally, calculating the probability consumes less energy, meaning the shopping cart needs to be recharged less frequently or at longer intervals, minimizing cart downtime.In other embodiments, a specific model can be a model specifically trained for a product group of the scanned product. In this case, the calculated probability comprises a plurality of probabilities, where each probability indicates how likely the at least one image of the inserted product is to match a product in the product group of the scanned product. Alternatively, or additionally, the specific model can be a model specific to a subset of a product group. In general, the trained model can be specifically trained for a set of products associated with the scanned product.

[0028] The trained model can be a neural network. For example, it can be a feedforward neural network, preferably a convolutional neural network. Training the specific model can be based on a specific image database. In particular, the model can be trained using an image database, preferably product-specific. For example, the model can be trained on a product-specific image database containing a large number of images of the product (preferably from different directions, angles, with varying lighting, shading, etc.). Alternatively, or additionally, the model can be trained on a specific image database for a different product.For example, the specific model can receive images from the product-specific image database as "positive examples" and images from a product-specific database of another product as "negative examples." Alternatively, or additionally, the negative examples can come from several different specific image databases.

[0029] The procedure may also include receiving a measured weight of the inserted product and comparing the measured weight with a target weight of the scanned product.

[0030] The target weight of the scanned product can be stored in a weight database and is a weight specific to that product. Comparing the measured weight with the target weight allows the scanned product to be compared with the product placed in the shopping cart with respect to another product-specific characteristic, thus providing an additional indicator of conformity. If the measured weight deviates from the target weight, it is unlikely that the scanned product matches the product placed in the shopping cart. The weight of the product placed in the shopping cart is a characteristic that complements the visual characteristics that can be assigned to the product through at least one image. This allows for differentiation between visually similar products that differ in weight.Similarly, products of similar weight that differ visually can be differentiated. This synergistic effect, resulting from the combination of visual and weight characteristics, leads to more reliable verification of the product placed inside and thus minimizes the possibility of customer deception. For example, a shopping cart and / or basket can be equipped with a scale that registers weight changes, allowing the weight of a newly placed product to be determined. The weight comparison can be performed, for example, by the shopping cart and / or its computer, which also receives at least one image and / or information about the scanned product (e.g., a tablet mounted on the shopping cart).

[0031] In other embodiments, further product-specific properties of the inserted product and / or target properties of the scanned product can be received. These can include sensor-based properties such as data from an infrared and / or heat sensor. Alternatively, or additionally, an RFID tag can be used.

[0032] This RFID tag can be used in addition to scanning the product to ensure that the inserted product is indeed the scanned product. The RFID tag data can be used before evaluating at least one image of the inserted product. The use of the RFID tag can be product- and / or product group-specific. For example, only specific products and / or specific product groups can be equipped with an RFID tag. Specific products and / or specific product groups might include, in particular, alcoholic products and / or high-priced products.

[0033] Receiving at least one image of the inserted product and / or selecting the trained model and / or determining the probability can be performed based on the assumption that the measured weight substantially matches the target weight.

[0034] The measured weight is considered to be in agreement with the target weight if the measured weight deviates from the target weight by no more than 20%, preferably no more than 10%. The tolerated deviation of the measured weight of the inserted product from the target weight of the scanned product can be a deviation specific to the scanned product. Alternatively, or additionally, the target weight can be a list of weights. For example, the list of weights can include moments of a weight distribution. For example, the list can include the first and second moments of a weight distribution, where the first moment is associated with the target weight and the second moment with the tolerated deviation. Alternatively, or additionally, a target weight can include multiple weights and / or multiple tolerances.For example, the target weight can include a first target weight with a first tolerance and a second target weight with a second tolerance.

[0035] By making the aforementioned process steps conditional on a substantial match of weights, a synergistic effect occurs in conjunction with the use of the image database specific to the scanned product and / or a specifically trained model. Comparing the weight of the inserted product with the target weight constitutes a first step in the verification process. If the two weights do not substantially match, the inserted product is highly unlikely to be the scanned product. Therefore, calling the trained model for verification is unnecessary, as the match has already been ruled out. This enables a rapid and conclusive falsification of the match between the inserted and scanned products, since comparing two weights can be done faster than evaluating at least one image of the inserted product with the trained model.Furthermore, the predictive power of the probability calculated by the trained model is increased.

[0036] In general, the inserted product can be identified with the scanned product if the calculated probability is greater than a threshold, preferably the threshold depending on the product group of the scanned product and / or the scanned product itself.

[0037] Identifying the inserted product with the scanned product involves equating the inserted product with the scanned product. In other words, the inserted product is considered the same as the scanned product. The threshold, which incorporates a probability, guarantees identification with a high degree of certainty; that is, the probability of misidentifying the inserted product with the scanned product is minimized, thus optimizing the verification of the inserted product. The threshold can be set at a probability of at least 80%, preferably at least 90%, of matching the scanned product with the inserted product. The threshold can be a value specific to the product group of the scanned product.A product's product group can comprise a set of products of similar nature, use, price level, and / or target audience. For example, the product group of a scanned alcoholic beverage can include a subset of all alcoholic beverages in the product range. The threshold for a subset of the alcoholic beverage product group can be higher than the threshold for a subset of the non-alcoholic beverage product group.

[0038] Alternatively, or additionally, the threshold can be a value specific to the scanned product itself. The product-specific threshold is individually defined for that product. In other implementations, a specific threshold can be defined for any subset of products. Generally, the thresholds are variable and can be dynamically changed. Similarly, product groups are variable and can be dynamically changed. For example, a price change for a product can lead to a change in the threshold and / or product group assignment. Furthermore, observed deceptive customer behavior can lead to a change in the threshold and / or product group assignment.

[0039] The procedure may also include the output of at least one image for manual verification of whether the inserted product should be identified with the scanned product if the probability is below a first threshold but above a second threshold.

[0040] Manual verification of at least one image can be performed by store staff. This involves manually comparing the image of the product with the scanned product. During this manual verification, the scanned product can be represented by a product name and / or an image associated with it. This verification can take place during the shopping process and / or at the end. For example, at the end of a shopping trip (with a full shopping cart), a store employee can use the image to check whether it actually shows the scanned product (without having to search for the product shown in the picture within the shopping cart).

[0041] Manual verification can take place if the probability is below a first threshold but above a second threshold. The first threshold is preferably below the threshold for identifying the inserted product with the scanned product. The first threshold can be less than a probability of 95%, preferably less than 90%. The first threshold can depend on the identification threshold. The second threshold is below the first threshold and can be greater than a probability of 60%, preferably greater than 70%. This defines a lower bound on the probability at which identification is still considered possible. In general, the second threshold can depend on the first threshold and / or the identification threshold.The first and second thresholds can be product-specific and / or product group-specific and / or variable and / or dynamic. Furthermore, the combination of the first and second thresholds results in a probability interval that includes all probabilities greater than the second threshold and less than the first threshold.

[0042] If the calculated probability lies within this interval, it can be stipulated, for example, that a manual inspection will be carried out with a probability of at least 30%, preferably 50%. The probability of inspection can be product- and / or product group-specific.

[0043] With a verification probability of at least 30%, preferably 50%, it is not necessary to manually check every product for which the probability is below the identification threshold. This relieves the burden on staff and reduces the number of personnel required. At the same time, the initial threshold guarantees a minimum level of security for the verification process. If the probability is below the initial threshold, a manual check can always be performed. If the probability is above the initial threshold, the inserted product can be identified with the scanned product. This allows, for example, for individual adjustments to the staffing situation and the risk of theft at each location.

[0044] In other embodiments, if the probability is below the second threshold, the inserted product can be considered different from the scanned product, and the two products cannot be identified. In this case, at least one image of the inserted product is always subjected to manual inspection. Alternatively, even in this case, at least one image can be subjected to manual inspection with a probability of at least 30%, preferably at least 50%. If the probability is below the first threshold, an error message can be displayed, and the customer's purchasing process can be interrupted until they have corrected the inserted product. The error message can include a request to remove the inserted product and insert the scanned product. This leads to a reduction in the number of personnel required to operate the store.In other cases, the shopping process can continue and a manual check will only take place at the exit of the market.

[0045] The need for a manual inspection can be communicated to staff via the shopping cart and / or their computer and / or the server and / or the cloud. For example, staff handheld devices (e.g., tablets) can be used for this purpose, displaying the shopping carts to be inspected. The image review can also take place on the staff's handheld devices and / or on the shopping carts' computers. In other cases, the need for a manual inspection can be indicated directly on the shopping cart itself, for example, by a corresponding light or similar indicator. In some cases, this indicator may only appear when the customer is about to complete their purchase, for example, when it is detected that the cart is approaching a checkout area of ​​a store. This detection can be made, for example, via Wi-Fi, Bluetooth, etc.

[0046] The method may further include scanning the scanned product, preferably with a scanner of the shopping cart and / or shopping basket, and / or weighing the inserted product, preferably by placing it in a weighing area of ​​the shopping cart and / or shopping basket, and / or creating at least one image of the inserted product, preferably with one or more cameras of the shopping cart and / or shopping basket.

[0047] Scanning a product identifies it. This scanning is preferably performed by a customer during the shopping process, for example, using a handheld scanner. However, the scanner can also be one or more scanners attached to a shopping cart, preferably positioned so that they automatically perform a scan when a product passes by. The scanned product is preferably one that can be placed in the shopping cart and / or basket. Scanning the product may involve scanning a barcode and / or a QR code. In general, scanning a product can include capturing a unique characteristic of that product.In other embodiments, scanning the scanned product can be based on creating at least one image with one or more cameras.

[0048] The weighing of the inserted product can be based on a weight difference within the weighing area, wherein the weight difference comprises a weight before the product is inserted and a weight after the product is inserted. The weighing area is that part of the shopping cart and / or shopping basket for which a weight change can be registered. The weighing area can include a shopping basket within the shopping cart. In other embodiments, the weighing area includes a shopping basket and at least part of a frame of the shopping cart. The weight difference is preferably determined when the shopping cart and / or shopping basket is stationary. The weighing of the weighing area can be dependent on the scanner being scanned. For example, the weighing area can be active only when the scanner is in use or a product is being scanned.By using a weighing area that is not continuously active, the cart's power consumption can be reduced, resulting in a longer operating period. This reduces energy costs and the cart's downtime. Alternatively, or additionally, the weighing area can be weighed continuously. Continuous weighing ensures that no product is added to the cart without being scanned. This guarantees that for every product placed in the shopping cart, a corresponding scan is performed, which then serves as the basis for verifying the product. Weighing can be carried out using load cells and / or force sensors.

[0049] The creation of at least one image by at least one camera on the shopping cart and / or basket can depend on the time of scanning and / or weighing. The at least one camera on the shopping cart continuously records at least part of the cart and saves the resulting image sequences to a storage device. If the weighing area of ​​the cart registers a weight change at any given time, a section of the image sequence is selected that encompasses the time of the weight change. This section can include images from the sequence before and / or after the weight change. For example, the section can begin at least i seconds, preferably at least 2 seconds, before the registered weight change and / or extend at most 10 seconds, preferably at most 5 seconds, beyond the time of the weight change.For example, the image data can be stored in a buffer memory that continuously holds image data from a specific period extending into the past.

[0050] By including the point in the image sequence where the weight changes, it is ensured that the inserted product is depicted in one of the images. This allows the trained model to be provided with suitable image data for verification. This reduces the likelihood of misidentifying the inserted product due to unsuitable image data, even though the inserted and scanned products are the same. This results in a more pleasant shopping experience for the customer, as the process is interrupted less frequently.

[0051] The fact that the image sequence includes the moment of the weight change also allows for differentiation between adding a new product and rearranging products already in the shopping cart and / or basket. For example, if products are rearranged in the shopping cart and / or basket, this appears to the camera to be a completely different product arrangement. In this case, however, the weight can be used as a reference, since rearranging the products does not change the weight.

[0052] Image sequences that do not belong to a section associated with a weight change are deleted from the memory. This ensures that only images relevant for verification are saved, preventing memory overflow. In particular, this allows the use of cost-effective storage, as the storage capacity does not need to cover the entire purchasing process. Alternatively, or additionally, the image sequence section can be selected to encompass the time of scanning.

[0053] Another aspect concerns a computer program for verifying a product placed in a shopping cart and / or shopping basket, comprising commands which, when the program is executed by a computer, cause it to perform the procedure described above.

[0054] The computer program can be executed on a computer associated with the shopping cart and / or shopping basket. Alternatively, or additionally, the computer program can be executed on a server and / or in a cloud, the server and / or cloud being associated with a store and / or a central office. In other embodiments, part of the computer program can be executed on a computer associated with the shopping cart and another part on a server and / or cloud associated with the store and / or central office.

[0055] Another aspect concerns a system for verifying a product placed in a shopping cart and / or basket. This system includes: means to receive at least one image of the placed product; means to select a trained model based on a scanned product; means to determine a probability that the at least one image of the placed product shows the scanned product, based on the trained model.

[0056] The verification system can be at least partially contained within a shopping cart. For example, the means for receiving the at least one image of the placed product and / or the means for selecting a trained model and / or the means for determining a probability can be part of the shopping cart, e.g., a computer (tablet) within the shopping cart. Preferably, the shopping cart includes the means for receiving the image, the means for selecting a trained model, and the means for determining a probability. Alternatively, parts of the system can be detached from the shopping cart and located separately in a store and / or a central location (e.g., a server and / or a central location, as described herein). In other embodiments, the system is located both on a shopping cart and separately in the store and / or a central location.For example, the shopping cart, the store, and / or the central office can all possess means for determining a probability. The selection of the means used for verification to determine a probability can depend on the scanned product. For example, the means for determining the shopping cart can be used for the first part of the product range, and the means for determining the store and / or the central office for the second part. The division of the product range into a first and second part can depend on the sales frequency and / or price level of the products. Alternatively, or additionally, the means for determining probability can be located on a server and / or in the cloud. In particular, the means for determining probability belonging to the store and / or the central office can be located on a server and / or in the cloud.

[0057] The means for receiving the image can be configured to receive data from a wireless network, which can be a WLAN, WMAN, WPAN, and / or WWAN network. In other examples, Bluetooth and / or a cellular connection (3G, 4G, 5G, etc.) can be used. Alternatively or additionally, the means for receiving the image can include physical connection elements, such as copper and / or fiber optic cables. The trained model can be a model specifically trained for the scanned product and can be trained based on an image database specific to that product.

[0058] The verification system may also include means for receiving a measured weight of the inserted product and means for comparing the measured weight with a target weight of the scanned product.

[0059] The means for receiving the measured weight and / or the means for comparison may be part of the shopping cart. Alternatively, the means for receiving the measured weight and / or the means for comparison may be separate means located in the store and / or at a central location. The means for receiving the weight may be at least partially the same as the means for receiving the image. The receiving means may be configured to receive data from a wireless network, which may be a WLAN and / or WMAN and / or WPAN and / or WWAN network. Alternatively, or additionally, the means for receiving the measured weight may include physical connection elements, such as copper and / or fiber optic cables. The means for comparison may be part of a computer and / or a processor.Furthermore, the means of comparison can at least partially coincide with the means of determining probability. For example, the means of comparison and the means of determining probability can be part of a computer, preferably a tablet, and / or a computing system.

[0060] Receiving at least one image and / or selecting the trained model and / or determining a probability can take place based on the fact that the measured weight is substantially similar to the target weight.

[0061] The verification system may further comprise at least one scanner for scanning the scanned product and / or a weighing device for weighing the inserted product and / or at least one camera for creating the at least one image.

[0062] The at least one scanner and / or weighing device and / or at least one camera can be part of the shopping cart. Preferably, the scanner, weighing device, and camera are part of the shopping cart. The weighing device can include load cells and / or force sensors. Preferably, several cameras are mounted on the shopping cart in such a way that at least some of the cameras optically capture the loading area of ​​the shopping cart.

[0063] Furthermore, the verification system may include means for outputting at least one image for manual verification as to whether the inserted product should be identified with the scanned product if the probability is below a first threshold but above a second threshold.

[0064] Alternatively, or additionally, at least one image can be output for manual inspection if the measured weight of the inserted product does not substantially match the target weight of the scanned product, but deviates within a certain tolerance range. Preferably, the output of at least one image also includes the measured weight of the inserted product and / or the target weight.

[0065] The means of dispensing can be part of the shopping cart and include a tablet computer. Alternatively, the means of dispensing can be separate, independent devices not part of the shopping cart. These separate means of dispensing can belong to a store and / or a central office. Preferably, the separate devices are a tablet computer and / or a computer.

[0066] Another aspect concerns a shopping cart or shopping basket with a system as described above. Furthermore, the shopping cart or shopping basket can have a memory, which at least partially comprises the image database and / or the trained model and / or the weight database.

[0067] The method and system described above are based on selecting a trained model based on a scanned product, specifically a specific trained model trained with a specific database. A simple setup of such a specific database and / or a simple provision of appropriately trained models is an important element for the practical applicability of the method. Secondly, a method and system are provided with which an image database and / or a model for verifying a product placed in a shopping cart and / or shopping basket can be automatically created or trained.

[0068] A method for automatically building an image database and / or automatically training a model to verify a product placed in a shopping cart and / or shopping basket may include the following steps: receiving a target weight of a scanned product; receiving a measured weight of the placed product; selecting at least one image of the placed product for inclusion in the image database and / or the trained model if the measured weight corresponds to the target weight.

[0069] Some steps of the process can be performed by a computer belonging to the shopping cart. For example, all steps of the process can be performed by the computer belonging to the shopping cart (e.g., a tablet mounted on the shopping cart). The selected image can then be sent to a server and / or a cloud (e.g., using the wireless communication methods described herein) so that it can be added to the corresponding image database for the scanned product and / or the trained model can be (further) trained for the scanned product.

[0070] Alternatively, or additionally, some steps of the procedure can be executed by a server and / or a cloud. For example, all steps of the procedure can be executed by the server and / or the cloud. In other embodiments, a first part of the steps can be executed by the shopping cart's computer and a second part by the server and / or the cloud. The first and second parts can be disjoint. For example, the shopping cart's computer can receive the measured weight of the shopping cart.

[0071] The measured weight is then transmitted to the server and / or cloud, which receives the measured weight of the product. The server and / or cloud also receives the target weight of the scanned product, preferably by retrieving it from a weight database. The server and / or cloud then selects at least one image of the product to be added to the image database. This image can, for example, be selected from an image sequence of the product transmitted by the shopping cart.

[0072] Alternatively, at least one image can be "selected" by having the server and / or the cloud request at least one image for the product placed in the shopping cart.

[0073] In general, the automated image database creation process can be performed independently and / or in conjunction with the verification process. An image database for verifying a product placed in a shopping cart is an image database that can be at least partially associated with a product. Partial association can include associating multiple products with an image database. Automated image database creation involves adding more images to an existing image database and / or associating at least one image with one or a group of products for the first time. Automated image database creation involves automatically associating at least one image with a product or a group of products.

[0074] In connection with the process for verifying a product placed in a shopping cart, accessing the aforementioned image database is already problematic, as it generally needs to be created first. This is often done manually, not automatically. One option for creating the image database is the centralized collection of product image data. However, this requires dedicated personnel, making the process very costly. Furthermore, many markets offer regional products for which centralized data collection is impractical, meaning that data must be captured either by the regional product's supplier or by staff in the regional market. Additionally, the lighting and environmental conditions under which the product is visually captured in the regional market differ from those under which it is captured centrally.This leads to an image database that does not reflect the realities of the market and thus results in an imprecise and / or unusable image recognition program. Alternatively, the image data can also be generated by staff in a store. However, this is also very costly and necessitates the continuous addition of new image data to the database, as new products are frequently added to the product range. According to the second aspect, the image database can be created and built by the customer during the purchasing process. By verifying the weights of the scanned and displayed products, incorrect image assignments can be ruled out with a very high degree of probability, allowing for the creation of a high-quality image database that also reflects the actual conditions in the market.As soon as customers scan a product and place it in their shopping cart, an image of the cart can be directly stored in the image database for that product, allowing the trained model to be further refined. This significantly reduces and / or eliminates errors caused by scanning an initial product and then incorrectly placing a second product.

[0075] The target weight of the scanned product is a weight associated with that product and can be a weight specific to that product. Generally, the target weight of the scanned product can be based on a weight database. A weight database can comprise a set of mappings, where a mapping assigns a target weight, preferably a product-specific target weight, to a product.

[0076] The measured weight of the inserted product is a weight measured during the insertion process. The target weight corresponds to the measured weight if their deviation does not exceed a predefined threshold. This threshold can be the same for all products and be a relative threshold. For example, the threshold can be at most 20%, preferably at most 10%, of the target weight of the scanned product. Alternatively, the threshold can be an absolute threshold, which is at most 100 grams, preferably at most 50 grams. In particular, the threshold can be at most 10 grams or at most 5 grams. Alternatively, or additionally, the threshold can depend on the sensitivity or resolution of the weighing device. In other embodiments, the threshold can be a product- or product-group-specific relative and / or absolute threshold.The product-specific threshold value can be stored in the weight database along with the product-specific target weight. The threshold value can be variable and / or dynamic. Selecting at least one image of the inserted product for inclusion in the image database involves associating that image with the scanned product or with a product group assigned to the scanned product. The image is preferably taken during the insertion of the product into the shopping cart and / or shopping basket. Similarly, selecting at least one image of the inserted product for inclusion in the trained model involves associating that image with the scanned product or with a product group assigned to the scanned product.The inclusion in the trained model can include the training of the trained model with respect to the included image, at least one image.

[0077] At least one image of the inserted product is selected for inclusion in the image database if the measured weight matches the target weight. This ensures that the weight of the inserted product corresponds to the weight of the scanned product. In particular, this increases the probability that the scanned product is indeed the inserted product. Thus, only those images that are highly likely to show the scanned product are selected for inclusion in the image database. This control mechanism guarantees high image database quality, resulting in a precisely trained model for verifying the scanned product. Specifically, this control mechanism enables the automatic building of the image database. If the measured weight does not match the target weight, at least one image cannot be selected.

[0078] The model can be trained in batches of images. Each batch contains a defined number of images. Once this number of images is reached in a batch, the model is trained on those images. This training can involve updating the model's previously trained parameters based on the images in the batch. Alternatively, the model can be trained based on at least some of the previously used training images and the batch itself. Batch training eliminates the need to train the model separately for each newly acquired image. This separate training is particularly problematic when many images need to be added to the model in a short period, as each training step consumes a certain amount of training time and computing resources.The phased training therefore relieves the computer resources, thus saving energy.

[0079] Training can take place, for example, on a server, a cloud, etc., that can be assigned to a market. Alternatively, training can be conducted regionally, nationally, and / or internationally for markets within a department store group, depending on whether the product is sold only regionally or nationally and / or internationally in a similar manner.

[0080] In other embodiments, the model can be trained based on the parameters trained up to the point of image acquisition and on the at least one image. This allows for fast and efficient integration of the image into the trained model, as the model does not need to be retrained with respect to all existing images. Alternatively, integration into the trained model can include training the model with respect to all previously acquired images together with the at least one image.

[0081] The image database can be an image database specific to the scanned product and / or the trained model can be a model specifically trained for the scanned product.

[0082] A product-specific image database contains only images exclusively associated with the scanned product. In other words, the product group assigned to the scanned product includes only that product. Similarly, a specifically trained model is a model associated only with the scanned product. The training of the specifically trained model can be based on this specific database. For example, the specifically trained model could be trained such that only images associated with the scanned product are labeled with a first label, and all images not associated with the scanned product are labeled with a second label.

[0083] Using product-specific image databases results in separate databases, each of which consumes less storage space than a global and / or product group-specific image database. This leads to more effective and efficient use, as a large database does not need to be loaded or sent. Furthermore, using a product-specific database in conjunction with a product-specific trained model creates a synergistic effect, since a specific database can already be used for training.

[0084] Similarly, using specifically trained models results in less complex and more memory-efficient models. In particular, these models can have a short execution time, thus providing a faster result—i.e., a probability—for verifying the product placed inside. This leads to a smoother and more pleasant shopping process, as the customer doesn't have to wait long for the verification result. At the same time, due to its specificity, the specifically trained model delivers accurate and meaningful results from which direct, conclusive inferences can be drawn.

[0085] Alternatively, or additionally, the specifically trained model can also be trained based on a specific image database that is specific to a different product. For example, the specific model can receive positive examples from the product-specific image database and negative examples from a product-specific database of a different product. Alternatively, or additionally, the negative examples can come from several different specific image databases.

[0086] The automated assembly process can further include outputting at least one image for manual verification of whether a recording should be made if the measured weight does not correspond to the target weight, wherein preferably a deviation of the measured weight from the target weight does not exceed a threshold value.

[0087] If the measured weight does not match the target weight within a tolerance defined by the threshold, at least one image is subjected to manual review. This manual review determines whether the image of the product placed in the cart matches the scanned product. If the manual review confirms a match, the image is added to the image database and / or the trained model. If no match is found, the image may be rejected. Manually selected images can be prioritized during training, as there is a higher probability that at least one image of the product placed in the cart will match the scanned product. Prioritization during training may involve assigning a higher weight to these images. The output of the at least one image can be displayed on the shopping cart itself, on a device in the store, and / or at a central location.The output at the shopping cart can occur during the shopping process, preferably based on the moment the product is placed in the cart, and / or at the end of the shopping trip. If output occurs at the end of the shopping trip, the output can include a variety of outputs. The device in the store is preferably a tablet computer. Furthermore, the output can be combined with a manual verification of the placed product. The threshold can be an absolute or relative threshold and can be product- or product-group-specific. The threshold can also be variable and / or dynamic. Alternatively, the output can be for manual verification in the event of a discrepancy between the two weights with a probability of at least 30%, preferably at least 50%.The output of at least one image may correspond to the output of at least one image in the procedure for verifying a product placed in a shopping cart or basket.

[0088] Manual verification ensures that at least one image of the inserted product matches the scanned product. This guarantees the development of a sufficiently large database, as images with a weight discrepancy are not immediately rejected for inclusion in the image database and / or the trained model. In particular, this method also accounts for manufacturer-initiated weight changes to a product, since the weight-based control mechanism is bypassed during manual verification. This can also contribute to updating the weight database.

[0089] The automated assembly method can further comprise scanning the product, preferably with a scanner of a shopping cart and / or shopping basket. Alternatively, or additionally, the method can comprise weighing the product, preferably by placing it in a weighing area of ​​the shopping cart and / or shopping basket, wherein the scanned product preferably corresponds to the product placed inside. Alternatively, or additionally, the method can comprise creating at least one image, preferably with at least one camera of the shopping cart and / or shopping basket.

[0090] Scanning a product identifies the product and is preferably performed by a customer. The scanned product is, in particular, a product that can be placed in a shopping cart and / or basket. Scanning the product may involve scanning a barcode, a QR code, and / or an RFID tag. Generally, scanning a product can involve capturing a feature unique to that product. In other embodiments, scanning the product may be based on creating at least one image with at least one camera.

[0091] The weighing of the inserted product can be based on a weight difference within the weighing area, wherein the weight difference comprises a weight before the product is inserted and a weight after the product is inserted. The weighing area is that part of the shopping cart and / or shopping basket for which a weight change can be registered. The weighing area can include a shopping basket within the shopping cart. In other embodiments, the weighing area includes a shopping basket and at least part of a frame of the shopping cart. The weight difference is preferably determined when the shopping cart and / or shopping basket is stationary. The weighing of the weighing area can be dependent on the scanner being scanned. For example, the weighing area can be active only when the scanner is in use or a product is being scanned.By using a weighing area that is not continuously active, the cart's power consumption can be reduced, resulting in a longer operating period. This reduces energy costs and the cart's downtime. Alternatively, or additionally, the weighing area can be weighed continuously. Weighing can be performed using load cells and / or force sensors.

[0092] The creation of at least one image by at least one camera on the shopping cart and / or basket can depend on the time of scanning and / or weighing. The at least one camera on the shopping cart continuously records a portion of the cart and saves the resulting image sequences to a storage device. If the cart's weighing area registers a weight change at any given time, a section of the image sequence is selected that encompasses the time of the weight change. This section can include images from the sequence before and after the weight change. For example, the section can begin at least i seconds, preferably at least 2 seconds, before the registered weight change and / or extend at most 10 seconds, preferably at most 5 seconds, beyond the time of the weight change.Because the image sequence includes the point in time of the weight change, it is ensured that the inserted product is depicted in one of the images within the sequence. This allows suitable image data to be supplied to the image database and / or the trained model for development. Image sequences that do not belong to a section associated with a weight change are deleted from memory. Thus, only images relevant for verification are stored, preventing memory overflow.

[0093] In particular, this allows the use of cost-effective storage, as the storage capacity does not need to be sufficient for the entire purchasing process. Alternatively, or additionally, the section of the image sequence can be selected to include the time of scanning.

[0094] In general, there can be a synchronization process for the trained models and / or image databases between the shopping cart and the server and / or cloud. This synchronization process can take place between the shopping carts and the server and / or cloud of a single store. Alternatively, or additionally, the synchronization process can occur between shopping carts from a large number of stores, preferably between a network of regional, national, and / or international stores. The synchronization process can be product-specific. This results in a large amount of image data for a specific product being generated quickly, as many shopping carts, potentially from different stores, contribute to the image data. Furthermore, the synchronization ensures that, upon completion of the process, all shopping carts have the same trained model.This ensures the quality of the trained model of each shopping cart for verifying a product placed in the shopping cart.

[0095] Another aspect concerns a computer program for the automated creation of an image database, comprising commands that, when the program is executed by a computer, cause it to perform the procedure described above.

[0096] Another aspect concerns a system for the automated creation of an image database and / or the automated training of a model to verify a product placed in a shopping cart and / or shopping basket. This system includes: means for receiving the target weight of a scanned product; means for receiving the measured weight of the placed product; means for selecting at least one image of the placed product for inclusion in the image database and / or the trained model if the measured weight matches the target weight.

[0097] The image database can be specific to the scanned product, and / or the trained model can be specifically trained for the scanned product. Furthermore, the target weight can be based on a weight database.

[0098] The automated assembly system may further include at least one scanner for scanning the scanned product and / or at least one weighing device for weighing the inserted product and / or at least one camera for creating at least one image.

[0099] The automated assembly system may further include means for outputting at least one image for manual verification of whether a recording should be made if the measured weight does not correspond to the target weight, and preferably if a deviation of the measured weight from the target weight does not exceed a threshold value.

[0100] Another aspect includes a shopping cart or shopping basket with a system as described above.

[0101] The shopping cart or shopping basket may also have a memory, the memory of which includes at least partially the image database, the trained model and / or the weight database.

[0102] Another aspect concerns a procedure for training a model to verify a product placed in a shopping cart and / or shopping basket. Training the model involves training using images selected using one of the procedures described herein. For example, a model specifically trained for a particular product can be trained using images of that product as "positive examples" and / or images of one or more other products as "negative examples," as described herein.

[0103] Finally, another aspect concerns a trained model and / or an image database for verifying a product placed in a shopping cart and / or shopping basket, which was trained and / or generated using images built with a method described herein.

[0104] Another aspect concerns a method for verifying a product placed in a shopping cart and / or shopping basket, wherein the method comprises: capturing a scanned product; tracking a movement of the scanned product during at least part of a placement process; determining, based on the tracking of the movement, whether the scanned product has been placed in the shopping cart.

[0105] The capture of the scanned product can involve optical capture of the product and can be performed by at least one camera and / or at least one camera system. Generally, the capture and / or the timing of the capture can be based on the moment the product is scanned.

[0106] Tracking the movement of the scanned product can be performed by a camera and / or a camera system. For example, tracking the movement can begin with the optical detection of the scanned product. For example, the camera and / or camera system can be configured to detect a product positioned next to a scanner and then track it within the image. Preferably, tracking the product's movement includes the product placement process and, optionally, placing the tracked product in the shopping cart and / or basket. For example, tracking the movement can end with the completion of the placement process. The completion of the placement process can include the removal of a hand, arm, and / or other limb from the shopping cart or basket. For example, the placement process of a product placed with a hand or arm can end when the product is in the shopping cart or basket.The tracking of movement is triggered when the product has been placed in the shopping cart and / or the hand or arm has left the shopping basket or cart. It is also possible that movement tracking is alternatively or additionally linked to the detection of the scanned product's weight in the shopping basket or cart. The scanned product can be tracked, for example, until a change in weight is detected. If this change matches the expected weight change for the scanned product, placement can be determined, at which point tracking can be terminated (optionally, after the removal of a hand or arm is detected). Determining whether the scanned product has been placed in the shopping cart is based on tracking the movement of the scanned product. This determination can be based on whether the optically detected product was placed in the shopping basket by the movement and / or whether the detected product remains in the shopping basket.For example, the scanned product cannot be considered placed in the shopping cart if it moves in and out of the shopping basket or cart, or if it does not move into the cart at all. Generally, determining whether the scanned product has been placed in the shopping cart does not require separate identification of the product, such as optical identification. In other embodiments, the method may include identification of the product, preferably based on a trained model, as described herein.

[0107] Although primarily described in relation to shopping carts and baskets, the procedures and systems explained herein can also be applied to stationary checkouts. For example, stationary checkouts can also have scanners, weighing functions, and / or cameras that are appropriately configured. Likewise, the data collected by stationary checkouts can be collected and / or processed analogously to how it is described herein for shopping carts and baskets.

[0108] In general, the concepts described with reference to shopping carts can also be applied to stationary checkouts.

[0109] A third aspect concerns a method for automatically building a weight and / or image database for product verification. This method involves obtaining an initial measured weight and / or a first captured image of the product from a stationary checkout. It also involves obtaining a second measured weight and / or a second captured image of the product from a shopping cart and / or basket. Furthermore, the method includes storing the first and second measured weights and / or the first and second captured images of the product in the weight and / or image database.

[0110] Receiving the first measured weight from the stationary checkout and receiving the second measured weight from the shopping cart can include receiving the first measured weight and receiving the second measured weight.

[0111] Additionally, or alternatively, receiving the first captured image of the product from the checkout and receiving the second captured image from the shopping cart can involve receiving the first captured image of the product from the checkout and receiving the second captured image from the shopping cart. Generally, the first measured weight can be received at a first time point, and the second measured weight can be received at a second time point. For example, the first and second time points can be different.

[0112] Receiving the first measured weight and / or the first image taken from the stationary cash register may include immediate receipt of the first measured weight and / or the first image taken from the stationary cash register.

[0113] For example, the first measured weight and / or the first captured image can be received directly from the stationary cash register. Alternatively, or additionally, receiving the first measured weight and / or the first captured image from the stationary cash register can involve receiving it indirectly. For example, a server and / or a data storage device can (initially) receive the first measured weight and / or the first captured image. The first measured weight and / or the first captured image can (then) be received from the server and / or the data storage device. Receiving the first measured weight and / or the first captured image from the server and / or the data storage device can involve the server and / or the data storage device forwarding the first measured weight and / or the first captured image.

[0114] Additionally, or alternatively, receiving the second measured weight and / or the second captured image from the shopping cart can involve directly receiving the second measured weight and / or the second captured image from the shopping cart. For example, the second measured weight and / or the second captured image can be received directly from the shopping cart. Alternatively, or additionally, receiving the second measured weight and / or the second captured image from the shopping cart can involve indirectly receiving the second measured weight and / or the second captured image from the shopping cart. For example, a server and / or a data storage device can (initially) receive the second measured weight and / or the second captured image. The second measured weight and / or the second captured image can (then) be obtained from the server and / or the data storage device.Receiving the second measured weight and / or the second captured image from the server and / or the data storage may involve the server and / or the data storage forwarding the second measured weight and / or the second captured image.

[0115] Generally, the checkout counter could be from one store, and the shopping cart could be from a second store. In other words, the first measured weight and / or first captured image of the product and the second measured weight and / or second captured image of the product could originate from different stores.

[0116] For example, the first department store may include a first server and / or first data storage, and the second department store may include a second server and / or second data storage.

[0117] Recording the first and second measured weights of the product in the weight database can involve merging them. For example, merging the first and second measured weights can also involve omitting the origin of the first and second measured weights—that is, whether the measured weight is obtained from the stationary checkout or the shopping cart—from the weight database. In other words, merging the first and second measured weights can involve treating them equally. Recording the first and second measured weights in the weight database can also involve storing the first and second measured weights of the product. For example, the first and second measured weights of the product can be stored in the weight database.Alternatively, or additionally, adding the first and second captured images to the image database can include merging the first and second images of the product. For example, merging the first and second images can include omitting the origin of the first and second images—that is, whether the captured image of the product was obtained from the checkout or the shopping cart—from the image database. In other words, merging the first and second images can involve treating them equally. Adding the first and second images to the image database can also include storing both the first and second images of the product. For example, the first and second images of the product can be stored in the image database.

[0118] The inventors recognized that existing approaches, where shopping cart providers offer systems and methods for collecting data from the carts, can be significantly improved by combining this data with data collected by stationary checkout systems. This allows for optimization of both data volume and quality, and enables the same datasets to be used and maintained across multiple markets or globally, regardless of the devices used in individual stores (stationary checkouts or smart shopping carts / baskets).

[0119] In general, a stationary checkout can be a self-service checkout (SCO) and / or a payment terminal. In some configurations, the stationary checkout can include means for weighing a product, e.g., using load cells. Additionally, or alternatively, the stationary checkout can include means for capturing an image, in particular a camera. Additionally, or alternatively, the stationary checkout can be an operator-operated checkout, i.e., a checkout that is typically operated by staff. An operator-operated checkout can also include means for capturing an image, e.g., a camera, and / or means for weighing a product, e.g., scales, e.g., with load cells. For example, the camera of an operator-operated checkout can be set up and / or positioned in such a way that it captures a (purchased) product.In some embodiments, the camera can be set up and / or arranged to capture the product from above, for example, automatically when the product is in a specific area of ​​the checkout. Capturing the product can include creating and / or generating image data of the product. In some embodiments, the staffed checkout includes a conveyor belt that automatically transports a product, for example, to a weighing area and / or an area where an image is automatically created. In some examples, it can also be provided that the staff moves a product accordingly.

[0120] In particular, the stationary cash register, e.g., the staffed checkout, can be configured to identify a product. For example, the stationary cash register can be configured to identify a product based on an image of the product. Identifying the product based on an image of the product can involve capturing an image of the product with a camera, e.g., a camera on the stationary cash register. For example, the stationary cash register can be configured to perform the procedures described herein for verifying a product. For example, the stationary cash register can include a conveyor belt. Specifically, the stationary cash register can include means for singulating products placed on the conveyor belt. In general, the stationary cash register, and in particular its camera, can be configured to identify the products placed on the conveyor belt.Additionally, or alternatively, the stationary cash register can be configured to record or weigh the products placed on the conveyor belt. For example, the stationary cash register can be configured to record or weigh the weight of each individual product placed on the conveyor belt and / or to identify each individual product placed on the conveyor belt based on the camera. In some embodiments, the stationary cash register can include a tunnel scanner.

[0121] Recording the first and second measured weights in the weight database allows for the inclusion of weights from both a cash register and a shopping cart. Recording weights from both cash registers and shopping carts maximizes the number of measured weights in the database. This maximizes the number of measured weights in the database, resulting in a richer and more detailed database that is more robust against erroneous data. Consequently, recording both the first and second measured weights in the weight database optimizes its quality.Similarly, including both the first and second captured images in the image database allows for the inclusion of product images captured by a checkout and a shopping cart. Including images from both maximizes the number of product images in the database. This maximizes the number of product images in the database, resulting in a richer and more detailed image database that is more robust against erroneous data, such as corrupted images. Consequently, including both the first and second captured images in the image database optimizes its quality. Generally, the first measured weight can be a weight measured by the checkout.Additionally, or alternatively, the first image captured can be an image taken by the stationary checkout. Additionally, or alternatively, the second measured weight can be a weight measured by the shopping cart. Additionally, or alternatively, the second image captured can be an image taken by the shopping cart.

[0122] For example, the first measured weight of the product may have been taken when the product was placed in and / or on the stationary checkout. Specifically, the measured weight may be specific to the product being placed in and / or on the checkout. For example, the weight of the product being placed in and / or on the checkout may have been measured for each product placed in and / or on the checkout. Additionally, or alternatively, the first image of the product being captured may have been taken when the product was placed in and / or on the stationary checkout. Specifically, the image being captured may be specific to the product being placed in and / or on the checkout. For example, the image of the product being placed in and / or on the checkout may have been captured for each product placed in and / or on the checkout. Similarly, the second measured weight of the product may have been taken when the product was placed in the shopping cart and / or shopping basket.

[0123] In particular, the second measured weight can be a weight specific to the product placed in the cart. For example, the weight of the product placed in the shopping cart and / or basket can have been measured for each product placed in the cart. Additionally, or alternatively, the second image of the product can be taken during the process of placing the product in the cart and / or basket. In particular, the image taken can be a weight specific to the product placed in the cart and / or basket. For example, the image of the product placed in the cart can have been taken for each product placed in the cart and / or basket.

[0124] Recording product weights and / or images into the weight and / or image database, as measured and / or captured by a checkout counter and / or shopping cart, allows for the inclusion of weights and / or images that correspond to the actual weight and / or appearance of the product. In other words, weights and / or images of the product can be included in the weight and / or image database exactly as measured and / or captured by the checkout counter and / or shopping cart. This enables the inclusion of measured weights and / or captured images that take into account the (real) environment, e.g., the lighting conditions in the store, the checkout counter, and / or the shopping cart.

[0125] The product may be a scanned product. For example, obtaining the first measured weight and / or the first captured image may include obtaining a product identifier. In particular, the identifier may be based, at least partially, on the scan of the scanned product. Additionally, or alternatively, obtaining the second measured weight and / or the second captured image may include obtaining a product identifier. In particular, the identifier may be based, at least partially, on the scan of the scanned product.

[0126] In general, the product identifier can include a feature specific to the product and / or characteristic of the product. In some embodiments, the product identifier can include an optically detectable feature. An optically detectable feature can be a feature that, for example, cannot be detected by the human eye but can be detected by cameras and / or general detectors. For example, the optically detectable feature can include a barcode and / or a QR code. Additionally, or alternatively, the optically detectable feature can include a digital watermark. For example, the digital watermark can be applied to the product using offset printing, laser printing, thermal printing, engraved printing, and / or flexographic printing. In general, the product can include multiple digital watermarks.For example, digital watermarks can be applied to different sides of the product. For instance, the digital watermark can be placed on opposite sides of the product. Applying and / or arranging digital watermarks on at least two sides of the product prevents (all) watermarks from being obscured, thus enabling the product's digital watermark to be detected securely and reliably. A digital watermark that cannot be detected by the human eye can, for example, include an invisible EAN code, such as EAN-8 and / or EAN-13, and / or an invisible QR code. The product being scanned can include a product whose barcode has been scanned. For example, the barcode can be part of a product identifier.In particular, product identification may be based, at least in part, on the product's identifier and / or barcode. For example, product identification may include unique product identification. In other words, the product's identifier and / or barcode may be configured in such a way that it uniquely identifies the product.

[0127] For example, the first measured weight and / or the first captured image of the product can include a measured weight and / or a captured image of a scanned product. The product can be scanned during the loading and / or placement process on and / or into the stationary checkout. Alternatively, or additionally, the second measured weight and / or the second captured image of the product can include a measured weight and / or a captured image of a scanned product. The product can be scanned during the placement process in the shopping cart and / or shopping basket.

[0128] Specifically, the scanned product can include a product scanned with a scanner at the stationary checkout. Alternatively, or additionally, the scanned product can include a product scanned with a scanner on the shopping cart and / or shopping basket.

[0129] For example, the scanner of a stationary cash register can be at least partially integrated into the cash register itself. Specifically, the stationary cash register can include the scanner. Alternatively, or additionally, the scanner of the stationary cash register can include a handheld scanner.

[0130] For example, the handheld scanner of the stationary cash register can be connected to the cash register, e.g., via a cable. Specifically, the scanner of the stationary cash register can scan the product during a loading and / or insertion process. For example, the scanner of the stationary cash register scans a barcode of the product during the loading and / or insertion process. Generally, the scanner of the stationary cash register can be configured to scan a storage area of ​​the stationary cash register. For example, the scanner can be positioned below and / or above the storage area. In some embodiments, the storage area can be associated with a weighing function; for example, the storage area can be configured to weigh a product placed on it.

[0131] Generally, the first and / or second weight measured may have been taken with a scale at the checkout counter. Additionally, or alternatively, the first and / or second weight may have been measured with a scale on the shopping cart. Additionally, or alternatively, the first and / or second image may have been taken with a camera at the checkout counter. Additionally, or alternatively, the first and / or second image may have been taken with a camera on the shopping cart.

[0132] In general, the procedure may also include determining a target weight of the product based at least partially on the first and / or second measured weight.

[0133] Alternatively, or additionally, the procedure can also include training a model based at least partially on the first and / or second recorded image.

[0134] In particular, the trained model can include a product-specific trained model. For example, the trained model can be a model trained to verify the product.

[0135] In general, determining a target weight for a product can be based on weights of the product already stored in the weight database. For example, determining the target weight can involve calculating the mean of the product's weights. Calculating a mean can include determining an arithmetic mean, a weighted mean, and / or a geometric mean. Alternatively, or additionally, determining a mean can involve evaluating a function, such as a polynomial. Determining the target weight can also involve determining a distribution, preferably a probability distribution. Alternatively, or additionally, determining the target weight can involve determining moments of the distribution, such as determining the distribution's variance. For example, different versions of a product with different weights (e.g.,(in the case of a product changeover and / or regional differences of a product), must be taken into account.

[0136] A specific image database can consist of images containing only those for a specific product. This specific image database can be a separate database for each product, distinct from the image databases of other products. Similarly, the trained model can be specifically trained for that product. This specifically trained model can, for example, be trained to recognize precisely that product, such as the one placed in the shopping cart and / or on the checkout counter. For this purpose, in addition to the image database for the product, images of other products can also be used as "negative examples" for training. In other embodiments, a specific model can be specifically trained for a product group. Alternatively, or additionally, the specific model can be specific for a subset of a product group.In general, the trained model can be specifically trained for a set of products associated with the product.

[0137] The trained model can be a neural network. For example, it can be a feedforward neural network, preferably a convolutional neural network. Training the specific model can be based, at least in part, on a specific image database. In particular, the model can be trained using the image database, which is preferably product-specific. For example, the model can be trained on a product-specific image database containing a large number of images of the product (preferably from different directions, angles, with varying lighting, shading, etc.). Alternatively, or additionally, the model can be trained on a specific image database for a different product.For example, the specific model can receive images from the product-specific image database as "positive examples" and images from a product-specific database of another product as "negative examples." Alternatively, or additionally, the negative examples can come from several different specific image databases.

[0138] Specifically, the process can also include transmitting the specified target weight of the product to the checkout and / or shopping cart. Alternatively, or additionally, the process can also include transmitting the trained model of the product to the checkout and / or shopping cart.

[0139] For example, the target weight of the product can be sent to the point-of-sale system and / or the shopping cart. Additionally, or alternatively, the trained model of the product can be sent to the point-of-sale system and / or the shopping cart. Generally, transmitting the specified target weight and / or the trained model may also include transmitting the product identifier.

[0140] Generally, the weight database can include a weight database specific to the product. Alternatively, or additionally, the image database can include an image database specific to the product.

[0141] For example, the product-specific weight database can contain only weight data for that specific product. In particular, the specific weight database can be a separate database for each product, distinct from the weight databases of other products. Alternatively, or additionally, a specific weight database can be a weight database specific to a product group. Alternatively, or additionally, the image database can be a product-specific image database. A specific image database contains only images for a specific product. The specific image database can be a separate database for each product, distinct from the image databases of other products. This allows for optimal on-site decisions (at the checkout and / or shopping cart) regarding the accuracy of a product's weight; e.g.,It can be checked whether a cheaper product was scanned, but then a more expensive one with a different weight is placed on top; similarly, for example, a recorded image of the product can be used to optimally check, based on the trained model / image database, whether the recorded image actually shows the correct product.

[0142] The automated setup process may further include outputting the first measured weight and / or the first captured image of the product for manual review. For example, the manual review may include checking whether the first measured weight and / or the first captured image should be entered into the weight and / or image database. Alternatively, or additionally, the automated setup process may further include outputting the second measured weight and / or the second captured image of the product for manual review. For example, the manual review may include checking whether the second measured weight and / or the second captured image should be entered into the weight and / or image database. In general, the output may include output to a user interface.Manual verification of the first weight and / or the first captured image of the product and / or the second weight and / or the second captured image of the product can be performed by the staff of a (shopping) store. This involves manually comparing the first / second measured weight and / or the first / second captured image of the product with a stored (ideal) weight and / or a stored (ideal) image of the product. The output of the first measured weight and / or the first captured image of the product can be displayed via a user interface. For example, the first measured weight and / or the first captured image can be displayed via a graphical user interface. Additionally, or alternatively, the output of the second measured weight and / or the second captured image of the product can also be displayed via a user interface.For example, the second measured weight and / or the second captured image can be displayed via a graphical user interface. If manual verification shows that the first / second measured weight and / or the first / second captured image of the product matches the stored (ideal) weight and / or (ideal) image of the product, then the first / second weight and / or the first / second image can be added to the weight and / or image database. Generally, the stored (ideal) weight and / or the stored (ideal) image of the product can be based on a scan of the product.

[0143] For example, the (ideal) weight and / or the (ideal) image of the product can be associated with an identifier, preferably an identifier based on the scan of the product.

[0144] Another aspect concerns a computer program for the automated creation of a weight and / or image database. The computer program includes commands which, when executed by a computer, cause it to perform the above-described procedure for the automated creation of a weight and / or image database.

[0145] Another aspect concerns a system for the automated creation of a weight and / or image database for product verification. The system includes means for obtaining an initial measured weight and / or a first captured image of the product from a stationary checkout, and means for obtaining a second measured weight and / or a second captured image of the product from a shopping cart and / or basket. The system further includes means for recording the first and second measured weights of the product in the weight database. Additionally, or alternatively, the system includes means for recording the first and second captured images of the product in the image database.

[0146] In general, the system may also include means suitable for carrying out the above-described procedure for the automated creation of a weight and / or image database for verifying a product.

[0147] The features described herein in relation to a process can each also be designed as features of a corresponding system, a computer program, a shopping cart, and / or a shopping basket, and vice versa. Furthermore, details of the first aspect can be combined with the second aspect, and vice versa. Details of the first and / or second aspect can also be combined with the third aspect, and vice versa.

[0148] 4. Brief description of the characters

[0149] Exemplary embodiments of the invention are described below with reference to the figures. The figures show:

[0150] Fig. 1A: Flowchart of a possible embodiment of the method for verifying the inserted product;

[0151] Fig. 1B: Flowchart of another embodiment of the method for verifying the inserted product;

[0152] Fig. 2A: Possible embodiment of a shopping cart with the system for verifying the inserted product and / or for automatically building an image database and / or trained model;

[0153] Fig. 2B: Exemplary representation of the optically accessible area of ​​a camera attached to a shopping cart; Fig. 3A: Flowchart of a possible embodiment of the method for automatic setup or automatic training;

[0154] Fig. 3B: Flowchart of another embodiment of the method for automatic assembly or automatic training;

[0155] Fig. 4: Flowchart of an exemplary embodiment of the method for the automated creation of a weight and / or image database for verifying a product;

[0156] Fig. 5: Exemplary schematic representation of a system for the automated creation of a weight and / or image database for verifying a product.

[0157] 5. Detailed description of preferred embodiments

[0158] Only a few possible embodiments of the invention are described in detail below. It should be understood that these exemplary embodiments can be modified and combined in various ways, provided they are compatible, and that certain features can be omitted where unnecessary.

[0159] Figure 1A shows a possible embodiment of a method 100 for verifying a product placed in a shopping cart and / or shopping basket. The method comprises receiving 110 at least one image of the placed product, selecting 120 a trained model based on a scanned product, and determining 130 a probability that the at least one image of the placed product shows the scanned product, based on the trained model.

[0160] The scanned product is one that has been captured by a Scanner 310 and can be uniquely identified by the scan. The scanning of the product preferably takes place before the product is inserted. Scanning and insertion preferably always occur in pairs; that is, a scan is always followed by insertion, and insertion is always preceded by a scan. Unique identification can be achieved through a feature unique to the scanner, such as a barcode, an EAN code, a GTIN code, an RFID tag, and / or a QR code.

[0161] In particular, the scanned product is one that will be billed to the customer at the end of the shopping process. Since the customer naturally wants the bill to be as low as possible, the scanned product may differ from the product placed in the shopping cart. For example, an inexpensive product might be scanned, while a more expensive one is placed in the shopping cart and / or basket. Therefore, it is not immediately clear that the customer will ultimately pay for the items actually in their shopping cart.

[0162] Receiving 110 of at least one image of the inserted product includes receiving images that were generated during the insertion process of the inserted product. The at least one image is preferably captured by a camera 330 of the shopping cart. The camera 330 continuously records, i.e., throughout the entire shopping trip, an area 335 of the shopping cart 300 that is optically accessible to the camera, wherein this area 335 comprises at least a part of the shopping cart 301. The at least one image is a video sequence of the insertion process and is selected from the continuous recording. The selection of the video sequence can depend on the time of scanning and includes a period before the time of scanning, the time of scanning, and a period after the time of scanning. The period before scanning, the time of scanning, and the period after scanning are temporally contiguous.As described herein, multiple cameras and / or camera systems can also be used on the shopping cart and / or shopping basket.

[0163] Selecting the trained model based on the scanned product involves choosing a model specifically trained on the scanned product. This trained model is an image recognition model based on neural networks. For example, a convolutional neural network can be used, in particular a YOLO Nano convolutional neural network (see, e.g., A. Wong, et al., 'YOLO Nano: a Highly Compact You Only Look Once Convolutional Neural Network for Object Detection', in 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurlPS Edition (EMC2-NIPS), Vancouver, BC, Canada, 2019, pp. 22-25. doi: 10.1109 / EMC2-NIPS53020.2019.00013). Selecting the trained model can, in particular, involve calling or loading an image recognition model from memory. Training such a model is based on appropriately prepared training data and a training phase.The training data prepared accordingly is based on the image database, which was obtained using the method for automatically building an image database.

[0164] Determining the probability that at least one image of the product placed in the cart shows the scanned product, based on the trained model for the scanned product, can be performed on a tablet computer belonging to the shopping cart. This determination is carried out by applying the trained model to the at least one image of the product placed in the cart. Since the trained model is specifically trained for the scanned product, the result of this determination only includes the probability that at least one image of the product placed in the cart shows the scanned product. Because the product billed to the customer corresponds to the scanned product, the scanner's information, i.e., the identification of the scanned product, can be used as preliminary information for image recognition.In other words, the image recognition program does not calculate the probability for all products represented in the product range, but only for the scanned product.

[0165] The identification process (130) can be performed for a first group of products on the shopping cart's tablet computer (320) and for a second group of products on a server and / or in the cloud. For example, the tablet computer (320) might only store the trained models for products in the first group. These products are frequently sold and therefore often perform the calculation (130). The tablet computer's storage capacity is limited, so not all trained models can be stored on it. The second group of products includes less frequently sold and / or high-priced products, for which very complex models are used. For example, the shopping cart could be connected to a wireless network and transmit at least one image of the product placed inside, as well as information about the scanned product, to a server and / or in the cloud.The information about the scanned product can be any type of information that allows the server and / or cloud to identify the scanned product. For example, the information about the scanned product can be the barcode of the scanned product and / or a pointer assigned to the scanned product. In this case, selecting the trained model can involve loading a model specifically trained for the scanned product. The probability calculation then takes place on the server and / or cloud. Once the probability calculation is complete, the result is sent back to the shopping cart via the wireless network.

[0166] If a product from the second group is scanned—that is, a product for which the probability is determined on a server and / or in the cloud—and a wireless connection between the shopping cart and the server and / or cloud cannot be established, the shopping cart will attempt to establish a connection at regular intervals. Once a connection is established, the data is transmitted. In this case, the shopping process is not interrupted, and the customer can continue shopping, as the probabilities for the placed products are only required at the end of the shopping process.

[0167] The process can further include identifying the inserted product with the scanned product. Identification occurs if the probability is greater than a threshold. The threshold can be set at a probability of at least 80%, preferably at least 90%. If the probability calculation is performed on the shopping cart's tablet computer, the product identification also takes place on the shopping cart's tablet computer. If the calculation takes place on a server and / or in the cloud, the identification can also be performed on the server and / or in the cloud, and the tablet computer is only notified via the wireless network whether or not identification is taking place. Alternatively, the identification can take place on the tablet computer based on the probability calculated on the server and / or in the cloud.

[0168] Procedure 100 can further include the output 240 of at least one image for manual verification of whether identification should occur. Output 240 occurs if the calculated probability is less than a first threshold but greater than a second threshold. The output can include a video sequence of the product placement process. Alternatively, the output can include individual images selected from the video sequence. The output can be displayed on the screen of the tablet computer 320. After the video sequence is displayed, manual input is requested from the tablet computer, which either identifies the placed product with the scanned product or defines the placed product as different from the scanned product. Alternatively, or additionally, output 240 can be displayed on a computer, preferably a tablet computer, which is managed by store personnel and / or a central office.The output can be generated separately for each product identification; that is, if a product is identified with a sufficiently high probability, the output occurs directly and immediately. Alternatively, the output can occur at the end of the purchasing process, preferably during the activation of a payment function and / or a payment transaction, and include the combined output of all products that were not identified with a sufficiently high probability.

[0169] Alternatively, the dispensing of item 240 can depend on the spatial position of the shopping cart within the store. For example, dispensing of item 240 can occur when the cart is pushed into a payment area of ​​the store, preferably located near the store exit.

[0170] Alternatively, or additionally, the payout of 240 can occur with a payout probability. If the calculated probability for the inserted product lies between the first and second bell values, the payout of 240 does not always occur, but only with a certain payout probability. This payout probability is at least 30%, preferably at least 50%. Alternatively, or additionally, the payout of 240 can always occur as soon as the calculated probability is less than the second bell value.

[0171] Figure 1B shows another embodiment of a method 200 for verifying a product placed in a shopping cart and / or shopping basket. The method 200 comprises receiving 210 a measured weight of the placed product and comparing 220 the measured weight with a target weight of the scanned product. Receiving 210 the measured weight is done in conjunction with weighing the placed product. The weighing of the placed product can be carried out by weighing devices 340a, 340b of the shopping cart. The target weight of the scanned product is part of a weight database, which includes target weights for all products in the product range. The weight database can be stored locally on the tablet computer 320 of the shopping cart 300, on a server, and / or in the cloud.Alternatively, or additionally, at least part of the weight database is located on the tablet computer 320 and another part on the server and / or in the cloud. The weight database can be dynamic, i.e., the target weights of the products represented in the database are changeable. The target weight of the scanned product can be the mean of a weight distribution of the scanned product. Furthermore, the target weight can include a variance of the weight distribution. In general, the method 200 can additionally include receiving 410 a target weight of a scanned product. Receiving 410 can occur before or after receiving 210, but preferably before comparing 220.

[0172] The comparison process (220) can take place on the tablet computer (320) or on a server and / or in the cloud. If the comparison process (220) takes place on the server and / or in the cloud, the shopping cart transmits the measured weight of the placed product and an identifier of the scanned product to the server and / or cloud via the wireless network. The identifier of the scanned product is configured so that the server and / or cloud can retrieve the corresponding target weight from the weight database. Generally, the comparison process (220) can include calculating the difference between the measured weight and the target weight.

[0173] If the measured weight matches the target weight, at least one image of the inserted product is received (110). For example, a video sequence is only selected if the two weights match. If the comparison (220) concludes that the two weights do not match, an error message can be displayed on the tablet computer (320). This error message prompts the customer to remove the inserted product from the shopping cart (301) and add a product corresponding to the scanned product to the shopping cart (301).

[0174] Figure 2A shows a possible embodiment of a shopping cart 300 with the system for verifying the product placed inside and the system for automatically building an image database and / or trained model. The shopping cart 300 comprises a shopping basket 301, a lower storage level 302, a chassis 303, a handheld scanner 310, a tablet computer 320, a camera system 330, and a plurality of load cells 340a, 340b. The handheld scanner 310 is connected to the tablet computer 320 and configured to scan the product. In other embodiments, the shopping cart may, for example, comprise exactly one load cell. The camera system 330 comprises a plurality of cameras (not shown) which capture at least a portion of the shopping basket 301. The cameras may be configured to create at least one image of the product placed inside. The optically accessible area 335 is the area of ​​the shopping cart captured by the cameras.Additionally, the camera system 330 can be configured to optically capture at least part of the lower storage level 302. Preferably, there is one camera for the shopping cart 301 and one camera for the lower storage level 302. In general, the cameras should be configured to have a frame rate of at least 24 fps, preferably at least 30 fps. The camera system is connected to the tablet computer 320.

[0175] The load cells 340a and 340b are connected to the tablet computer and configured to register weight changes in the shopping cart 301 and / or the lower storage level 302. Specifically, the load cells can be configured to weigh a product placed in the shopping cart 301 and / or the lower storage level 302. Generally, the load cells for registering weight changes in the shopping cart 301 can be different from those for registering weight changes in the lower storage level 302. The load cells are configured to resolve weight changes with an accuracy of at least 5 grams, preferably at least 2 grams. Generally, the accuracy of comparing the measured weight with the target weight can depend on the accuracy of the load cells.

[0176] Furthermore, the shopping cart 300 and / or the tablet computer 320 have interfaces for sending and receiving data via a wireless connection, preferably a WLAN connection. The data can be sent to a server and / or a cloud. The tablet computer 320 comprises memory and processors, with at least part of the image database and / or part of the weight database stored on the memory of the tablet computer 320. The memory of the tablet computer is, for example, at least 5 GB, at least 50 GB, at least 100 GB, or preferably at least 128 GB.

[0177] Fig. 2B shows a portion of the optically detectable area 335 of the shopping basket 301 of the shopping cart 300 from the perspective of a camera of the camera system 330, which is mounted near the tablet computer 320 and / or on the tablet computer 320 itself. In particular, Fig. 2B shows the process of placing a product P into the shopping basket 301. Fig. 3A shows a possible embodiment of a method 400 for the automated creation of an image database and / or for the automated training of a model to verify a product placed in a shopping cart 300 and / or shopping basket. The procedure includes receiving 410 a target weight of a scanned product, receiving 420 a measured weight of the inserted product, and selecting 430 at least one image of the inserted product for inclusion in the image database and / or the trained model if the measured weight corresponds to the target weight.

[0178] Preferably, process 400 takes place simultaneously with process 200 and is executed by the customer during the purchasing process. The scanned product is preferably a product scanned with scanner 310. The target weight of the scanned product is based on a weight database. Receiving the measured weight of the inserted product (420) takes place during an insertion process and is preferably performed by load cells 340a and 340b. In general, process step 420 can be identical to process steps 210.

[0179] Figure 3B shows another possible embodiment of a method 500 for the automated creation of an image database and / or the automated training of a model to verify a product placed in a shopping cart 300 and / or shopping basket. The method comprises receiving 410 a target weight of the scanned product and receiving 420 a measured weight of the placed product. If the measured weight does not correspond to the target weight, at least one image 510 is output for manual verification as to whether a photo should be taken. The output of the at least one image of the placed product can include a video sequence of the placement process. The output can be displayed on the tablet computer 320 of the shopping cart 300 and / or on a computer associated with the store and / or a central location, preferably a tablet computer.Output 510 for manual inspection can at least partially coincide with output 240. If a recording is to be made after output 510, then at least one image of the inserted product is selected (430) for inclusion in the database and / or the trained model. In particular, the identification of the inserted product with the scanned product during output 240 can lead to the selection (430) of at least one image of the inserted product for inclusion in the image database and / or the trained model. If the measured weight corresponds to the target weight, then selection (430) occurs directly, without output 510.

[0180] Figure 5 shows an exemplary embodiment of a method 600 for the automated creation of a weight and / or image database for verifying a product. The method 600 comprises obtaining 610 a first measured weight and / or a first captured image of the product from a stationary cash register.

[0181] For example, Receiving 600 can include receiving the weight of the product measured by the stationary cash register and / or receiving an image of the product captured by the stationary cash register. The weighed and / or captured product can include a product that has been placed on the stationary cash register and / or inserted into the stationary cash register. In some embodiments, Receiving 610 the first measured weight and / or the first captured image can include receiving the first measured weight and / or the first captured image. In some embodiments, the first measured weight and / or the first captured image can be received by the stationary cash register. For example, the stationary cash register can send the first measured weight and / or the first captured image.In particular, the POS system can send the first measured weight and / or the first captured image to a server and / or cloud suitable for recording the first measured weight and / or the first captured image in the weight and / or image database. Alternatively, or additionally, the POS system can send the first measured weight and / or the first captured image to a server and / or cloud suitable for forwarding the first measured weight and / or the first captured image. For example, the first measured weight and / or the first captured image can be forwarded to a server and / or cloud suitable for recording the first measured weight and / or the first captured image of the product in the weight and / or image database.

[0182] Method 600 further comprises obtaining 620 a second measured weight and / or a second captured image of the product from a shopping cart and / or a shopping basket. For example, obtaining 620 may include obtaining a weight of the product measured from the shopping cart and / or the shopping basket and / or obtaining an image of the product captured by the stationary checkout. The weighed and / or captured product may include a product that has been placed in the shopping cart and / or the shopping basket. In some embodiments, obtaining 620 the second measured weight and / or the second captured image may include receiving the second measured weight and / or the second captured image. In some embodiments, the second measured weight and / or the second captured image may be received from the shopping cart and / or the shopping basket.For example, the shopping cart and / or shopping basket can send the second measured weight and / or the second captured image. Specifically, the shopping cart and / or shopping basket can send the second measured weight and / or the second captured image to a server and / or cloud service suitable for storing the second measured weight and / or the second captured image in the weight and / or image database.

[0183] Alternatively, or additionally, the shopping cart and / or basket can send the second measured weight and / or the second captured image to a server and / or cloud suitable for forwarding this information. For example, the second measured weight and / or the second captured image can be forwarded to a server and / or cloud suitable for adding the product's weight and / or image to the weight and / or image database.

[0184] Procedure 600 further comprises recording 630 the first and second measured weights of the product in the weight database and / or recording 630 the first and second captured images of the product in the image database. For example, recording 630 the first and second measured weights of the product may include saving the first and second measured weights in the weight database.

[0185] Alternatively, or additionally, capturing the first and second images of the product may include saving them to the image database. Generally, capturing the first and / or second measured weight may also include formatting the data.

[0186] Alternatively, or additionally, the capture process (630) can include formatting the first and / or second captured image. The capture process (630) can also include merging with existing weight and / or image data. For example, the first and second measured weights of the product can be merged with existing weight data for the product, such as previously received weight data. Alternatively, or additionally, the first and second captured images of the product can be merged with existing image data for the product, such as previously received image data. In general, the capture process (630) of the first and / or second measured weight into the weight database can include associating the first and / or second measured weight with a timestamp. Alternatively, the receive process (610) and / or the receive process (620) can include receiving a timestamp.In other words, the first measured weight and / or the first captured image and / or the second measured weight and / or the second captured image can be obtained with a timestamp. For example, the timestamp can include a point in time that corresponds to the measurement of the first and / or second weight. For instance, the first measured weight and / or the first captured image can be associated with a first timestamp. Specifically, the first timestamp can correspond to the point in time at which the checkout counter weighed and / or recorded the product. Alternatively, or additionally, the second measured weight and / or the second captured image can be associated with a second timestamp. Specifically, the second timestamp can correspond to the point in time at which the shopping cart and / or basket weighed and / or recorded the product.

[0187] Additionally, the method 600 may further comprise determining 642 a target weight of the product based at least partially on the first and / or the second measured weight. For example, determining 642 the target weight of the product may comprise determining a mean of the product's weights. Determining a mean of weights may include determining an arithmetic mean, a weighted mean, and / or a geometric mean. In general, determining a mean may include evaluating a function, e.g., evaluating a function based on the product's weights. In some embodiments, determining the mean may include determining a distribution of the product's weights. Determining a distribution of the product's weights may include determining moments of the weight distribution.In general, determining the target weight of the product can be based on at least one weight already recorded in the weight database. For example, determining the target weight can be based on at least five, preferably at least 50, and most preferably at least 100 weights already recorded in the weight database. In some embodiments, determining the target weight can be based on a timestamp associated with the weight. For example, determining the target weight can be based on weights of the product already recorded in the weight database such that a timestamp associated with the weights is compatible with a limit value. A timestamp compatible with a limit value can include the fact that a time difference between the timestamp and a time associated with determining the target weight is less than the limit value.

[0188] Alternatively, or additionally, the procedure 600 can include training 644 a model based at least partially on the first and / or second recorded image.

[0189] For example, training the model can also be based on images of the product already stored in the image database. Training can involve training a neural network. In particular, the model can be trained to recognize the product.

[0190] Alternatively, or additionally, the procedure 600 can include transmitting 646 the specified target weight to the stationary checkout and / or the shopping cart and / or the shopping basket. For example, the transmission 646 of the specified target weight can include transmitting an average value. Additionally, or alternatively, the transmission 646 can include transmitting moments of a distribution, preferably a probability distribution. Alternatively, or additionally, the trained model can also be transmitted to the stationary checkout and / or the shopping cart and / or the shopping basket.

[0191] Alternatively, or additionally, the method 600 can include outputting 648 of the first measured weight and / or the first captured image of the product for manual review to determine whether it should be entered into the weight and / or image database. Alternatively, or additionally, the method 600 can include outputting 648 of the second measured weight and / or the second captured image of the product for manual review to determine whether it should be entered into the weight and / or image database. For example, the output 648 can include outputting the first weight and / or the first captured image of the product to a user interface, preferably a graphical user interface. Additionally, or alternatively, the output 648 can include outputting the second weight and / or the second captured image of the product to a user interface, preferably a graphical user interface.Figure 5 shows an example of System 700 for the automated creation of a weight and / or image database for product verification. System 700 includes means for obtaining a first measured weight and / or a first captured image of the product from a stationary checkout. System 700 further includes means for obtaining a second measured weight and / or a second captured image of the product from a shopping cart and / or shopping basket. System 700 further includes means for storing the first and second measured weights and / or the first and second captured images of the product in the weight and / or image database.

[0192] The means of receiving 710, 720 can be configured as separate means of receiving. Alternatively, the means of receiving 710, 720 can be configured as a single means of receiving. In other words, a single means of receiving can be configured to receive both the first measured weight and / or the first captured image of the product and the second measured weight and / or the second captured image. For example, the means of receiving 710, 720 can include a transceiver. In particular, the transceiver can be configured for bidirectional communication, for example, via one or more antennas, wired and / or wireless connections. In some embodiments, the transceiver can include a (wireless) transmitter / receiver. In particular, the transceiver can communicate bidirectionally with another (wireless) transceiver. The transceiver can include a modem for modulating data packets.The modem can be configured to forward the modulated data packets to one or more antennas for transmission and / or to demodulate the data packets received by one or more antennas.

[0193] In some embodiments, the system 700 further comprises at least one stationary cash register 740a, 740b, 740c. The at least one stationary cash register can be configured to communicate with the receiving means 710. The at least one stationary cash register 740a, 740b, 740c can be arranged in different retail outlets. For example, the stationary cash register 740a can be arranged in a first retail outlet and the stationary cash registers 740b, 740c in a second retail outlet. The at least one stationary cash register 740a, 740b, 740c can include at least one load cell and / or at least one camera. Preferably, the at least one load cell and the at least one camera are arranged such that the camera captures an area weighed by the at least one load cell.The at least one stationary cash register 740a, 740b, 740c may be set up to perform steps of procedure 100, 200 to verify a product placed on the stationary cash register.

[0194] The system may further comprise at least one shopping cart and / or shopping basket 750a, 750b, 750c. For example, the at least one shopping cart and / or shopping basket 750a, 750b, 750c may be configured to perform steps of procedure 100, 200 for verifying a product placed in a shopping cart and / or shopping basket.

[0195] The receiving means 730 can comprise at least one processor 734. The receiving means 730 can further comprise at least one memory and / or a storage medium 738. The at least one processor 734 and the at least one memory and / or the at least one storage medium 738 can be configured for electrical communication with each other. Alternatively, the at least one processor 734 and the at least one memory and / or the at least one storage medium 738 can be coupled to each other in some other way, for example, operationally, communicatively, functionally, electronically, or via at least one bus. The at least one memory can comprise random-access memory (RAM) and / or read-only memory (ROM). The at least one memory 738 can comprise machine-readable and / or machine-executable code.For example, the memory may contain code which, when executed by the 724 processor, causes the 700 system to execute the procedure for automatically building a weight and / or image database.

Claims

January 12, 2026 KBST GmbH K177738WO ANE / Snj REQUIREMENTS 1. Method (600) for the automated creation of a weight and / or image database for verifying a product, comprising: Receipt (610) of an initial measured weight and / or a first recorded image of the product from a stationary cash register; Obtaining (620) a second measured weight and / or a second recorded image of the product from a shopping trolley and / or a shopping basket; Recording (630) the first and second measured weight and / or the first and second recorded image of the product into the weight and / or image database.

2. Method (600) for automated assembly according to claim 1, wherein: the first measured weight includes a weight measured by the stationary cash register and / or the first recorded image includes an image recorded by the stationary cash register; and / or the second measured weight includes a weight measured from the shopping cart and / or shopping basket and / or the second recorded image includes an image taken from the shopping cart and / or shopping basket.

3. Method (600) for automated assembly according to claim 1 or 2, wherein the product is a scanned product.

4. Method (600) for automated assembly according to claim 3, wherein: The acquisition (610) of the first measured weight and / or the first captured image includes the acquisition of an identifier of the product; preferably wherein the identifier is based at least partially on the scan of the scanned product; and / or The obtaining (620) of the second measured weight and / or the second recorded image includes obtaining an identifier of the product; preferably wherein the identifier is based at least partially on the scan of the scanned product.

5. Method (600) for automated assembly according to claim 3 or 4, wherein: the scanned product includes a product scanned with a scanner at the stationary checkout; and / or The scanned product includes a product scanned with a scanner of the shopping cart and / or shopping basket.

6. Method (600) for automated assembly according to any one of claims 1 to 5, wherein: the first and / or the second measured weight was measured with a scale at the stationary checkout and / or the shopping cart; and / or the first and / or the second recorded image was taken with a camera at the stationary checkout and / or the shopping cart.

7. Method (600) for automated assembly according to any one of claims 1 to 6, further comprising: Determining (642) a target weight of the product based at least partially on the first and / or second measured weight; and / or training (644) a model based at least partially on the first and / or second recorded image; preferably wherein the trained model comprises a product-specific trained model; most preferably wherein the trained model is a trained model for verifying the product.

8. Method (600) for automated assembly according to claim 7, further comprising: Transmitting (646) the specified target weight of the product to the stationary checkout and / or the shopping cart; and / or Transmit (646) the trained model of the product to the stationary cash register and / or the shopping cart.

9. Method (600) for automated assembly according to any one of claims 1 to 8, wherein: the weight database is a weight database specific to the product; and / or The image database is a product-specific image database.

10. Method (600) for automated assembly according to any one of claims 1 to 9, further comprising: Output (648) of the first measured weight and / or the first captured image of the product for manual review to determine whether a recording should be made; and / or Output (648) of the second measured weight and / or the second recorded image of the product for manual review to determine whether recording should be made; preferably wherein the output includes output to a user interface.

11. Computer program for the automated creation of a weight and / or image database, comprising commands which, when the program is executed by a computer, cause it to execute the method according to any one of claims 1 to 10.

12. System (700) for the automated creation of a weight and / or image database for verifying a product, comprising: Means of obtaining (710) a first measured weight and / or a first recorded image of the product from a stationary cash register; Means of obtaining (720) a second measured weight and / or a second recorded image of the product from a shopping trolley and / or a shopping basket; Means for recording (730) the first and second measured weight and / or the first and second recorded image of the product into the weight and / or image database.

13. System for automated assembly according to claim 12, further comprising means for carrying out the method according to any one of claims 2 to 10.

Images