Bagging support device, bagging support method, and program

Abstract

A bagging assistance device according to the present invention comprises: a product information acquisition means for acquiring product information of purchased products of a customer; a classification means for classifying, on the basis of the product information, the purchased products into groups representing a bagging order for placing the purchased products into a bag for taking the purchased products home; an image acquisition means for acquiring, before or during a bagging operation, images taken of the purchased products that are stored in a basket or a cart prior to being bagged; an identification means for identifying the bagged purchased products on the basis of the images; and a display control means for displaying, on a display unit, the purchased products included in at least one group among the classified groups on the basis of the bagged purchased products and classification results.

Description

Bagging support device, bagging support method, and recording medium

[0001] The present disclosure relates to a technique for assisting bagging.

[0002] Patent Document 1 describes a technique for notifying a customer of the storage position and registration order of each purchased product for storing the purchased product in a storage container based on product information.

[0003] Japanese Patent Application Laid-Open No. 2019-174878

[0004] However, with the information processing device described in Patent Document 1, the person packing the bags must search for the products in the order in which they were registered.

[0005] An example of an objective of the present disclosure is to provide a technology that can assist in bagging purchased items.

[0006] A bagging support device in one aspect of the present disclosure includes a product information acquisition means for acquiring product information of products purchased by a customer, a classification means for classifying the purchased products into groups representing the order in which the purchased products will be placed into bags for carrying the purchased products home based on the product information, an image acquisition means for acquiring images of the purchased products contained in a basket or cart before and during the bagging process and before they are bagged, an identification means for identifying the bagged purchased products based on the images, and a display control means for displaying the purchased products included in at least one of the classified groups on a display unit based on the bagged purchased products and the classification results.

[0007] In one aspect of the present disclosure, a bagging assistance method involves a computer acquiring product information for products purchased by a customer, classifying the purchased products based on the product information into groups that represent the order in which the purchased products will be placed into bags for carrying the purchased products home, acquiring images of the purchased products stored in a basket or cart before and during the bagging process, identifying the bagged purchased products based on the images, and displaying on a display unit the purchased products included in at least one of the classified groups based on the bagged purchased products and the results of the classification.

[0008] In one aspect of the present disclosure, a recording medium records a program that causes a computer to execute the following processes: acquire product information about products purchased by a customer; classify the purchased products based on the product information into groups that represent the order in which the purchased products will be placed into bags for carrying the purchased products home; acquire images of the purchased products stored in a basket or cart before and during the bagging process, before being bagged; identify the bagged purchased products based on the images; and display the purchased products included in at least one of the classified groups on a display unit.

[0009] One example of the effect of the present disclosure is that it can assist in bagging purchased items.

[0010] FIG. 1 is a block diagram showing the configuration of a bagging support device in an embodiment. FIG. 2 is a diagram showing an example of a classification result in an embodiment. FIG. 3 is a diagram showing an example of a display screen in an embodiment. FIG. 4 is a diagram showing an example of a display screen in an embodiment. FIG. 5 is a diagram showing an example of a display screen in an embodiment. FIG. 6 is a flowchart showing an example of the operation of a bagging support device in an embodiment. FIG. 7 is a block diagram showing the configuration of a bagging support device in a first modified example. FIG. 8 is a block diagram showing the configuration of a bagging support device in a second modified example. FIG. 9 is a diagram showing a hardware configuration in which a bagging support device in the present disclosure is realized by a computer device and its peripheral devices.

[0011] Embodiments of the present disclosure will be described in detail with reference to the drawings.

[0012] In this disclosure, bagging refers to the process of transferring purchased items from a container, such as a basket or cart, used in the store to another container, such as a bag, basket, or other container for takeout. Note that the containers used in the store and the containers for takeout are not limited to these examples.

[0013] 1 is a block diagram showing the configuration of a bagging support device 100 according to an embodiment. Referring to FIG. 1, the bagging support device 100 includes a product information acquisition unit 101, a classification unit 102, an image acquisition unit 103, an identification unit 104, and a display control unit 105.

[0014] In the embodiment, bagging assistance when a customer performs self-scanning shopping in a store will be described. In self-scanning shopping, the customer uses a terminal device. The terminal device has a camera for scanning barcodes, a display for displaying a list of purchased items and product prices, and a communication function for communicating with a POS (Point of Sales) server (not shown). The terminal device may be connected to a scanner for scanning barcodes. The terminal device may be a POS terminal device attached to a cart or basket, or a mobile terminal owned by the customer.

[0015] In self-scanning shopping, customers scan the barcodes of products while shopping around the store. The terminal device used for self-scanning shopping then registers the scanned products as purchased products. The customer then places the scanned products in a cart or basket.

[0016] In an embodiment, the bagging support device 100 may be included in a terminal device used for self-scan shopping. Alternatively, the bagging support device 100 may function as a server and be connected to the terminal device via a network. Furthermore, the functions of the bagging support device 100 may be realized by processing between the terminal device and the server.

[0017] Next, the configuration of the bagging support device 100 in this embodiment will be described in detail.

[0018] 1, a product information acquisition unit 101 is an example of an acquisition unit for acquiring product information of products purchased by a customer. The product information acquisition unit 101 acquires product information of each product purchased by the customer in a store.

[0019] The product information includes information representing indicators used by the sorting unit 102 (described later) to classify the purchased products into groups representing the order of bagging. For example, the indicators include product size, product weight, and product load capacity. These indicators may be expressed as specific numerical values ​​or several levels. For example, product size may be expressed in 5 cm increments. Furthermore, product weight and product load capacity may be expressed in 10 g increments or as "large," "medium," "small," or the like. Note that examples of indicators expressed in several levels are not limited to these. The sorting unit 102 extracts information representing one or more indicators of product size, product weight, and product load capacity from the product information. Here, the product load capacity refers to the weight that a product can withstand when another product is placed on top of it while remaining in good condition. A good product condition refers, for example, to a state in which the product is not crushed or damaged. Furthermore, the product information may further include information indicating attributes of the purchased products, such as product shape, product category, product fragility, product odor intensity, or product storage temperature, as indicators used by the sorting unit 102 to classify the purchased products into groups representing the order of bagging. The shape of the product may be, for example, a rectangular parallelepiped, cube, or sphere, or whether it has thorns. The category of the product may be, for example, food or daily necessities, fresh food or processed food, or detergent, paper products, or other daily necessities. The fragility, odor intensity, and storage temperature of the product may be expressed as specific numerical values ​​or several levels. The product information may also include the name of the purchased product or a product image representing the purchased product. The product image may be an image of the purchased product taken in advance, an image of the product extracted from an image acquired by the image acquisition unit 103 (described later), or an illustration that can identify the product. The product information may also include information indicating the price of the product and a limit on the number of products that can be purchased, as information to be used in the payment of the purchased product.

[0020] For example, the product information acquisition unit 101 acquires product information of the purchased product from a product information database provided outside or inside the bagging assistance device 100, using product identification information for identifying the purchased product scanned during self-scan shopping. The product identification information is included in the barcode beforehand. The camera of the terminal device or a scanner connected to the terminal device acquires the product identification information by reading the barcode.

[0021] The timing at which the product information acquisition unit 101 acquires product information for purchased products is not particularly limited. For example, the product information acquisition unit 101 may acquire product information while shopping, i.e., before checkout. Furthermore, for example, the product information acquisition unit 101 may acquire only product information from the product information database while shopping, such as product name and product price, that is used for product registration or checkout related to product purchases. In this case, the product information acquisition unit 101 may acquire product information to be used for bagging assistance using product identification information before starting the bagging assistance operation. The product information to be used for bagging assistance includes the size, weight, and load capacity of the product.

[0022] The sorting unit 102 is an example of a sorting means that sorts purchased items into groups that represent the order in which the purchased items are to be put into bags for carrying them home, based on the acquired product information. The sorting unit 102 sorts the purchased items into groups that represent the order in which the purchased items are to be put into bags, based on the above-mentioned indices. For example, the sorting unit 102 sorts the purchased items into groups that represent the order in which the purchased items are to be put into bags, using the size, weight, and load capacity of each purchased item. Two examples of specific processing methods for sorting will be described later. In the following description, the groups that represent the order in which the purchased items are to be put into bags, as sorted by the sorting unit 102, may also be referred to as bagging groups.

[0023] First, we will explain bagging groups. Bagging groups are groups of purchased items that are packed and overlapped in a bag, classified according to the desired position, from bottom to top, in the bag so that the purchased items can be delivered in good condition. The sorting unit 102 classifies the desired positions, from bottom to top, in the bag based on the aforementioned index. The bagging process is smoother if the purchased items are bagged in order, starting with the bottom ones. Therefore, bagging groups are ordered such that a group of purchased items that should be bagged in the bottom tier is the group of purchased items that should be bagged first, and a group of purchased items that should be bagged above the previous group is the group of purchased items that should be bagged after the group of purchased items that should be bagged first. The group of purchased items that should be bagged last is either a group of purchased items that should be bagged in the top tier or a group of purchased items that can be bagged anywhere in the bag. Purchased items that can be bagged anywhere in the bag are, for example, small and therefore can be bagged anywhere, and are light and will not damage other purchased items. In other words, a bagging group is, for example, a group of purchased items that should be bagged first, a group of purchased items that should be bagged after the group of purchased items that should be bagged first, and a group of purchased items that should be bagged last. The number of groups of purchased items that should be bagged after the group of purchased items that should be bagged first is not limited. That is, the number of groups representing the bagging order is two or more. For example, if the number of groups of purchased items that should be bagged after the group of purchased items that should be bagged first is one, i.e., if the number of bagging groups is three, the group of purchased items that should be bagged first may be group 1, the group of purchased items that should be bagged after the group of purchased items that should be bagged first may be group 2, and the group of purchased items that should be bagged last may be group 3.

[0024] The sorting unit 102 may determine the number of groups to be bagged based on the number of purchased items or the number of types of purchased items. In this case, a minimum limit on the number of purchased items to be included in a group may be set in advance. This is because, when a group contains a large number of purchased items, it is less likely that all of the purchased items to be removed from the basket or cart before bagging will be in a location that is difficult to remove from the basket or cart. This is because, when a group contains a large number of purchased items, it is more likely that a customer will select an item that is easy to remove from the group of purchased items to be removed. In other words, when a group contains a small number of purchased items, it is more likely that all of the purchased items to be removed from the basket or cart before bagging will be in a location that is difficult to remove from the basket or cart. This is because, when a group contains a small number of purchased items, it is more likely that a customer will have difficulty removing the purchased items to be removed from the group.

[0025] If the group of purchased products to be packed is 1, the sorting unit 102 may decide not to sort the purchased products into a group in the order in which they should be packed.

[0026] FIG. 2 is a diagram illustrating an example of the results of classification by the classification unit 102. The classification results include information identifying the groups, the order in which each group is to be bagged, and the purchased items included in each group. For example, the classification results include information indicating that Group 1, which is the group of purchased items to be bagged first, contains three items A, one item B, and one item C. The same applies to Group 2, which is the group of purchased items to be bagged after the group of purchased items to be bagged first, and Group 3, which is the group of purchased items to be bagged last. The classification results are not limited to the example shown in FIG. 2. For example, the number of groups may be other than three. Furthermore, the number of purchased items included in a group may be other than three or four. Furthermore, the information identifying the purchased items may be product names or product identification information. Furthermore, the classification results may include product information and images of the purchased items.

[0027] Next, two examples of specific classification processing methods performed by the classification unit 102 will be described.

[0028] An example of a classification processing method is a processing method in which the order in which the purchased items are to be packed into bags is determined, the determined order is divided, and each divided purchased item is classified into a group.

[0029] The sorting unit 102 first determines the order in which the purchased items will be bagged. The sorting unit 102 determines the order based on the size, weight, and load-bearing capacity of the items, which are examples of indices used for classification. For example, the sorting unit 102 determines the order in which the purchased items will be bagged so that items with larger size, weight, and load-bearing capacity are bagged first. Furthermore, which index to prioritize in determining the order may be predetermined. For example, if the size of a product is x1, the weight of the product is x2, and the load-bearing capacity of the product is x3, the sorting unit 102 calculates y = a x1 + b x2 + c x3 (a, b, and c are weighting values) for each purchased item. The sorting unit 102 may determine the order in which the purchased items will be bagged by sorting each purchased item in order of the magnitude of the y value. Alternatively, the sorting unit 102 may determine the order in which the purchased items will be bagged so that items with smaller load-bearing capacity and size are bagged first.

[0030] The method for determining the bagging order and the product information used to determine the bagging order are not limited to this example, and any known method may be used. For example, as a method for determining the bagging order, the sorting unit 102 may determine the positions of the purchased products by further using the dimensions of the bags acquired by the bag information acquisition unit 206 (described later). For example, the sorting unit 102 determines the combination of positions within the bag based on the product information and combination conditions. The combination conditions may include, for example, placing products with larger size, weight, and load capacity at the bottom according to the bag dimensions, or, in a combination of products arranged vertically, the load capacity of the lower product exceeds the weight of the upper product. The combination conditions may also include not placing a product on top of a fragile product, or, in a combination of products arranged nearby, ensuring that the difference in storage temperature between the products is within a predetermined value. The sorting unit 102 may determine the bagging order by starting with the product determined to be at the bottom.

[0031] Next, the sorting unit 102 divides the determined order into groups of purchased products. The sorting unit 102 may, for example, determine the division of the order so that each group contains an equal number of purchased products. Furthermore, for example, the sorting unit 102 may determine the division of the order based on a large difference in the size, weight, or load capacity of the purchased products between the previous and next purchased products.

[0032] Another example of a classification method is a method of classifying the purchased items according to threshold values ​​for product size, product weight, and product load capacity. For example, the classification unit 102 classifies the purchased items into each bagging group so that each purchased item conforms to the number of bagging groups and the upper and lower limits for product size, product weight, and product load capacity set for each bagging group. The upper and lower limits for product size, product weight, and product load capacity for each bagging group may be determined in advance according to the number of groups.

[0033] Although examples of the classification method have been described above, the classification method is not limited to these examples. For example, the classification unit 102 may classify the purchased items into groups according to the order of bagging, based on the positions at which the purchased items are to be placed as determined above, and according to the height at which the purchased items are to be placed in the bags.

[0034] The image acquisition unit 103 is an example of an image acquisition means that acquires images of purchased items stored in a basket or cart before and during the bagging work. The image acquisition unit 103 acquires images of purchased items stored in a basket or cart before being bagged from a camera (not shown).

[0035] The camera may be, for example, a camera on a terminal used for self-scanning shopping, a surveillance camera installed in a store, or a camera installed on a bagging workbench in a store. The camera captures an image of the area including the basket or cart from above. Here, above refers to the open side of the basket or cart. In other words, an image of the purchased items contained in the basket or cart before being bagged will show the purchased items at the top of the basket or cart. Items at the top are items that are easiest for customers to pick up.

[0036] First, the image acquisition unit 103 acquires an image of the purchased items contained in the basket or cart before they are bagged. The image before the bagging refers to the time before the customer has completed product registration and payment and has begun the task of packing the purchased items into bags for takeout. The image before the bagging is captured by the camera after product registration is completed or payment is completed, and the image acquisition unit 103 acquires the captured image. For example, when the self-scan shopping process progresses to the payment process, the image acquisition unit 103 instructs the camera to take an image. In response to the instruction received from the image acquisition unit 103, the camera captures an image of the purchased items in the basket or cart before they are bagged, and transmits the captured image to the image acquisition unit 103.

[0037] Furthermore, the image acquisition unit 103 acquires images of the purchased items stored in the basket or cart before being bagged during the bagging work. The bagging work period is the period from when the customer completes product registration and payment until the customer finishes the work of packing the purchased items into bags for takeout, or the period from when the customer starts the work of packing the purchased items into bags for takeout until the customer finishes the work.

[0038] The image acquisition unit 103 may acquire images during the bagging operation at predetermined time intervals. The predetermined time interval may be, for example, every 1 second or every 0.1 seconds. The image acquisition unit 103 may be a camera that captures images, or a communication unit connected to a camera and acquiring the captured images from the camera. The image acquisition unit 103 connected to a camera may also have a control function for controlling the camera's capture and image transmission. If the image acquisition unit 103 is a camera, the image acquisition unit 103 captures images at predetermined time intervals. If the image acquisition unit 103 is a communication unit connected to a camera and acquiring the captured images from the camera, the image acquisition unit 103 may acquire the images captured at predetermined time intervals as needed. Similarly, if the image acquisition unit 103 is a communication unit connected to a camera and acquiring the captured images from the camera, the image acquisition unit 103 may acquire images at predetermined time intervals from continuously captured images. If the image acquisition unit 103 has a control function for controlling the camera's capture and image transmission, the image acquisition unit 103 may acquire images at predetermined time intervals by causing the camera to capture and transmit the images at predetermined time intervals.

[0039] Furthermore, for example, the image acquisition unit 103 may acquire images at the timing when there is a change in the images of the purchased items in the basket or cart during the bagging work. For example, the camera detects movement of an object in the basket or cart within the range captured by the camera itself. The camera may then transmit images captured before and after the detected movement to the image acquisition unit 103. Here, the objects related to the images that are desired to be acquired during the bagging work are the purchased items and the hands of the customer bagging them. The camera detects movement in the purchased items or the hands. The camera may then transmit images captured before and after the movement to the image acquisition unit 103.

[0040] Furthermore, the image acquisition unit 103 may further acquire images of take-out bags and the like during the bagging work.

[0041] The identification unit 104 is an example of an identification means that identifies bagged purchased items based on an acquired image. The bagged purchased items identified by the identification unit 104 are purchased items that have been transferred from a basket or cart used for shopping around the store into a bag or the like for takeout. The identification unit 104 identifies bagged purchased items based on at least one of the acquired images, which are an image of the basket or cart containing the purchased items before bagging and an image of the bag or the like containing the bagged purchased items. Identifying bagged purchased items using an image of the basket or cart containing the purchased items before bagging has the advantage that the purchased items in the basket or cart are more likely to appear in the image than the purchased items contained in the bag or the like. Identifying bagged purchased items using an image of the bag or the like containing the bagged purchased items has the advantage that purchased items that have been removed from the basket or cart but not bagged can be avoided from being identified as bagged purchased items.

[0042] First, a case will be described in which an image of a basket or cart containing purchased items before bagging is used to identify bagged purchased items. The identification unit 104, for example, compares two images captured in chronological order and identifies purchased items that are included in the earlier captured image but not in the later captured image as bagged purchased items. Purchased items that are included in the earlier captured image but not in the later captured image are presumed to have been moved from the basket or cart. In other words, the identification unit 104 may identify purchased items that have been removed from the basket or cart as bagged purchased items. Note that the two chronologically captured images do not need to be two images captured consecutively, as long as they are in chronological order.

[0043] Furthermore, the identification unit 104 may identify, for example, purchased items included in a group of purchased items to be bagged next after the purchased item identified as bagged, which purchased items are included in the image captured earlier but are not included in the image captured later, as bagged purchased items. There are two groups of purchased items to be bagged next after the purchased item identified as bagged. One is a group of purchased items including the purchased item identified as bagged when there are purchased items that are included in the same group as the purchased item identified as bagged but are not identified as bagged. The other is a group of purchased items to be bagged after the group of purchased items identified as bagged when there are no purchased items that are included in the same group as the purchased item identified as bagged but are not identified as bagged. Note that these two groups of purchased items are examples of groups of purchased items that are currently to be bagged. In other words, the identification unit 104 may identify, as bagged purchased items, purchased items that are included in the group of purchased items to be currently bagged, among purchased items that are included in the image captured earlier but are not included in the image captured later. The method by which the identifying unit 104 identifies the group of purchased products that should currently be packed into bags will be described later.

[0044] Conversely, the identification unit 104 may identify, among purchased items that are included in the image captured earlier but not included in the image captured later, purchased items that are included in a group of purchased items that should be bagged after the group of purchased items that should currently be bagged, as temporarily placed purchased items rather than bagged. Temporarily placing refers to a customer temporarily placing an upper purchased item on a bagging work table or the like in order to retrieve an lower purchased item in a basket or cart.

[0045] These identification methods are believed to improve the accuracy of identifying bagged purchased items. For example, if a purchased item that is not part of the group of purchased items to be currently bagged is hidden by another purchased item that has changed position in the basket or cart, the identification unit 104 can be prevented from identifying the hidden purchased item as a bagged purchased item.

[0046] Furthermore, the identification unit 104 may identify temporarily placed products based on the images acquired by the image acquisition unit 103, the classification results, and information on the purchased products identified as bagged. For example, the identification unit 104 identifies, as temporarily placed products, purchased products that are placed at the top of a basket or cart included in an image captured during the bagging process and that are included in a group of purchased products that should be bagged after the group currently being bagged. Because the image is captured from above the basket or cart, purchased products that are placed at the top of the basket or cart are the purchased products that appear in the image. In the following description, purchased products that are placed at the top of a basket or cart and that are included in a group of purchased products that should be bagged after the group currently being bagged will be referred to as temporarily placed products.

[0047] A more specific example of the process of identifying temporarily placed products by the identification unit 104 will be described. For example, the identification unit 104 identifies purchased products shown in an image. Then, based on information about the purchased products identified as bagged, the identification unit 104 identifies which group of the classification results corresponds to the group of purchased products that should currently be bagged. Based on the classification results and information about the identified group of purchased products that should currently be bagged, the identification unit 104 identifies whether the identified purchased product is included in the group of purchased products that should currently be bagged or in the group of purchased products that should be bagged later. If the purchased product shown in the image is included in the group of purchased products that should be bagged later, the identification unit 104 identifies the purchased product as a temporarily placed product. The process of identifying temporarily placed products is not limited to this example.

[0048] The method by which the identification unit 104 identifies a group of purchased items to be currently bagged will be described. The identification unit 104 identifies a group of purchased items to be currently bagged based on the purchased items identified as already bagged and the classification results. More specifically, the identification unit 104 first determines whether there are any purchased items identified as already bagged. If it is determined that there are no purchased items identified as already bagged, i.e., if the customer has not yet started bagging, the identification unit 104 identifies the group of purchased items to be currently bagged as the group of purchased items to be initially bagged. Next, if it is determined that there are any purchased items identified as already bagged, the identification unit 104 determines whether there are any purchased items not yet determined to be bagged in the bagging group to which the purchased item identified last bagged belongs. If there is a purchased item that belongs to the same group as the purchased item identified last bagged and has not yet been identified as bagged, the identification unit 104 identifies the same group as the purchased item identified last bagged as the group of purchased items to be currently bagged. Furthermore, if there are no purchased products that belong to the same group as the purchased products identified as being packed in bags and that have not yet been identified as being packed in bags, the identifying unit 104 identifies the group of purchased products that should be packed next to the group of purchased products identified as being packed in bags as the group of purchased products that should currently be packed in bags.

[0049] Another example of a method by which the identification unit 104 identifies bagged purchased items based on an image will be described. For example, the identification unit 104 may identify a purchased item held by a customer in an image as a purchased item removed from a basket or cart. For example, the identification unit 104 may extract images that include the customer's hand from images acquired at a predetermined time interval. The identification unit 104 may then identify the purchased item being held by the customer's hand from the extracted images. This example of the identification method may also be applied, as in the above-described identification method, to a method of increasing the accuracy of identification based on a group of purchased items currently to be bagged, a method of using an image of a bag or work table in addition to a basket or cart, or a method of identifying temporarily placed items. Note that the method of identifying bagged purchased items is not limited to these.

[0050] Furthermore, when the identification unit 104 identifies all purchased items included in a bagging group classified by the classification unit 102 as bagged purchased items, the identification unit 104 may identify that bagging is complete for that group. Furthermore, for example, the terminal device may include a means for receiving an input indicating completion of bagging for each bagging group, and the means may receive an input indicating completion from the customer, whereby the identification unit 104 may identify that bagging of the purchased items included in the bagging group is complete. Information about a group identified as having completed bagging may be used for display control by the display control unit 105, which will be described later.

[0051] The identification unit 104 may determine that all of the purchased items have been bagged. For example, the identification unit 104 may determine that all of the purchased items have been bagged when the image of the basket or cart does not include any purchased items. Furthermore, for example, the identification unit 104 may determine that all of the purchased items have been bagged when a comparison result between the bagged purchased items identified by the identification unit 104 and the purchased items acquired by the product information acquisition unit 101 matches. Note that the identification unit 104 may also determine that a comparison result between the bagged purchased items identified by the identification unit 104 and the purchased items acquired by the product information acquisition unit 101 does not match. If the comparison results do not match, there is a possibility that the identification of the bagged purchased items or the product registration of the purchased items has not been performed accurately.

[0052] Next, a case where an image of a bag containing the bagged purchased items is used to identify the bagged purchased items will be described. Furthermore, for example, if the image acquisition unit 103 acquires an image of a carry-out bag or the like during the bagging process, the identification unit 104 may identify the bagged purchased items based on the image. The identification unit 104 may identify the bagged purchased items in the same manner as the method for identifying purchased items removed from a basket or cart. For example, the identification unit 104 may compare two images captured in chronological order and identify the purchased items that are not included in the earlier captured image but are included in the later captured image as the bagged purchased items. This allows the identification unit 104 to accurately identify the bagged purchased items even when the purchased items are removed from the basket or cart but not put into a bag. An example of a case where a purchased item is removed from the basket or cart but not put into a bag is when a customer temporarily places a purchased item above the purchased item on a bagging work table or the like in order to remove a purchased item that is vertically lower in the basket or cart.

[0053] The display control unit 105 is an example of a display control means that causes the display unit to display purchased items included in at least one of the groups classified based on the bagged purchased items and the classification results. The classification results are the classification results by the classification unit 102, as illustrated in FIG. 2 . The display control unit 105 causes the display unit to display each group and the purchased items included in each group. The display control unit 105 causes the display unit to display purchased items included in each group that have not been identified as bagged. The display control unit 105 may also cause the display unit to display at least purchased items included in the group of purchased items currently to be bagged, among the purchased items included in each group. Alternatively, the display control unit 105 may cause the display unit to display purchased items included in the group of purchased items currently to be bagged and purchased items included in the group of purchased items that should be bagged after the group of purchased items currently to be bagged. The group of purchased items to be bagged after the group of purchased items currently to be bagged may be only the group of purchased items to be bagged immediately after the group of purchased items currently to be bagged, or part or all of the group of purchased items to be bagged after the group of purchased items currently to be bagged. In addition, the display control unit 105 may also cause the display unit to display a group of purchased items that have already been bagged.

[0054] The display control unit 105 first displays each group on the display unit based on the classification results before the start of the bagging work. Fig. 3 is an example of a display screen before the start of the bagging work. Details of Fig. 3 will be described later. The display control unit 105 may display the group of purchased items that are currently to be bagged in a way that is easy for the customer to recognize. The method of displaying the purchased items and bagging groups in a way that is easy for the customer to recognize is not limited to the example of Fig. 3.

[0055] The display control unit 105 may change the display of the purchased items identified as being bagged to indicate that the bagging of the purchased items has been completed. FIG. 4 is an example of a screen indicating that the bagging of the purchased items has been completed. Details of FIG. 4 will be described later. The display control unit 105 may change the display by, for example, changing the display color of the items that have been bagged or deleting the items that have been bagged, so that the customer performing the bagging operation can recognize the purchased items that have already been bagged.

[0056] Furthermore, the display control unit 105 may change the display of the group for which bagging has been completed, by receiving information from the identification unit 104 about the group for which bagging has been completed, to indicate that all purchased items included in the group have been bagged. FIG. 5 is an example of a screen indicating that bagging of purchased items included in bagging group 1 has been completed. Details of FIG. 5 will be described later. The display control unit 105 may change the display by, for example, changing the display color of the bagged group for which bagging has been completed or deleting the bagged items, so that a customer performing the bagging operation can recognize the bagged group that has already been bagged.

[0057] When the identification unit 104 determines that all purchased items have been bagged, the display control unit 105 may cause the display unit to display a screen indicating that bagging is complete. Furthermore, when the identification unit 104 determines that the comparison result between the bagged purchased items identified by the identification unit 104 and the purchased items acquired by the product information acquisition unit 101 does not match, the display control unit 105 may cause the display unit to display that there are items that were not scanned or items that were scanned excessively. Furthermore, for example, when the comparison result between the bagged purchased items identified by the identification unit 104 and the purchased items acquired by the product information acquisition unit 101 does not match, the bagging assistance device 100 may send a call notification to a store clerk's terminal, etc. This allows the store clerk to respond to product registration errors or shoplifting. If the comparison result between the bagged purchased product identified by the identification unit 104 and the purchased product acquired by the product information acquisition unit 101 does not match, the display control unit 105 may display a screen instructing the customer to input information to confirm that there are no problems with product registration and bagging. This prevents the customer from being inconvenienced by a store clerk being notified if there is an error in identifying the bagged purchased product.

[0058] The display control unit 105 may also display guidance to temporarily store purchased items that are placed at the top of the basket or cart included in the image captured during the bagging process and that are included in a group of purchased items that should be bagged after the group currently being bagged. Purchased items that are placed at the top of the basket or cart included in the image captured during the bagging process and that are included in a group of purchased items that should be bagged after the group currently being bagged are the temporarily stored items described above. The temporarily stored items are identified by the identification unit 104.

[0059] For example, when the identification unit 104 identifies a temporary storage item, the display control unit 105 may display a guide to temporarily store the temporary storage item. Furthermore, when all purchased items included in a group of purchased items to be currently bagged are identified as temporary storage items, the display control unit 105 may display a guide to temporarily store the temporary storage items.

[0060] 6 is an example of a screen displaying a guide to temporarily place a purchased product that is a temporary placement product. In FIG. 6, a guide saying "Please temporarily place a cabbage" and a product image of a cabbage surrounded by a frame are displayed. These are examples of a guide to temporarily place a purchased product. In this way, the display control unit 105 may display the purchased product identified by the identification unit 104 as a temporary placement product and the guide to temporarily place the temporary placement product in a manner that is recognizable to the customer.

[0061] The operation of the bagging support device 100 configured as above will be described with reference to the flowchart of FIG.

[0062] FIG. 7 is a flowchart showing an outline of the operation of the bagging support device 100 in this embodiment.

[0063] As shown in FIG. 7, first, the product information acquisition unit 101 acquires product information of the product purchased by the customer (step S101).

[0064] Next, the sorting unit 102 sorts the purchased products into groups according to the order of packing based on the product information acquired in step S101 (step S102).

[0065] Next, the display control unit 105 causes the display unit to display the group of purchased items currently to be bagged, as identified by the identification unit 104 (step S103). The display control unit 105 causes the display unit to display the group of purchased items currently to be bagged, based on the classification results and information indicating the identified group of purchased items currently to be bagged. The example screen in FIG. 3 is a screen displayed before the customer starts the bagging operation. In FIG. 3, the screen displays each bagging group, which is the classification result by the classification unit 102, and images or product names of the purchased items, which is information indicating the purchased items in each bagging group. The example shown in FIG. 3 includes bagging group 1, bagging group 2, and bagging group 3. Each group also includes purchased items. Before the customer starts the bagging operation, bagging group 1 is displayed large, and the purchased items in bagging groups 2 and 3 are displayed only as images, while the purchased items in bagging group 1 are displayed as images and product names. In this way, the display control unit 105 may display the purchased items included in the group of purchased items currently to be bagged so that the customer can easily recognize them. Also, bagging group 1 is displayed larger on the screen than bagging groups 2 and 3.

[0066] Next, the image acquisition unit 103 acquires an image of the purchased items contained in the basket or cart before being bagged (step S104).

[0067] Next, the identifying unit 104 identifies the bagged purchased products based on the image acquired in step S104 (step S105).

[0068] Next, the identification unit 104 determines whether the packing of the purchased items has been completed (step S106). If the packing has not been completed, the packing support device 100 continues the operations from step S103 to step S106 until the packing is completed. If the operation proceeds from step S106 to step S103, the display control unit 105 may change the display of the purchased items or the group of packing orders according to the information of the purchased items identified as being packed.

[0069] In step S103, when the operation proceeds from step S106 to step S103, the display control unit 105 changes the display of the purchased items, for example, based on the information of the purchased items identified as having been bagged. An example of the change in display of the purchased items is shown in FIG. 4. In FIG. 4, two 1 L bottles of milk are displayed in gray. Compared to FIG. 3, which shows an example of the display screen before the start of the bagging work, in FIG. 4, the display of the two 1 L bottles of milk identified as having been bagged by the identification unit 104 has been changed. This is an example of a change in display indicating that the two 1 L bottles of milk have already been bagged. The method of changing the display of purchased items identified as having been bagged is not limited to this example.

[0070] In step S103, when the operation proceeds from step S106 to step S103, the display control unit 105 may, for example, change the display of the group in the bagging order based on information about the group identified as having completed bagging. In FIG. 5 , bagging group 1 is displayed in gray. Also in FIG. 5 , bagging group 2, which is a group of purchased items to be bagged after bagging group 1, is displayed larger than bagging groups 1 and 3, and bagging group 1 is displayed smaller than bagging group 2. Compared to FIG. 3 , which shows an example of a display screen before the start of the bagging operation, FIG. 5 shows a change in the display of the group in the bagging order identified by the identification unit 104 as having been bagged. These are examples of display changes indicating that the bagging of the purchased items included in bagging group 1 has been completed. The method of changing the display of the group in the bagging order identified as having been bagged is not limited to this example.

[0071] After the operations of steps S103 to S106 are repeated, the identifying unit 104 determines that the packing of the purchased items has been completed (step S106). If it is determined that the packing has been completed, the packing support device 100 ends the series of operations.

[0072] In the bagging support device of the present embodiment described above, the product information acquisition unit acquires product information about purchased products, and the classification unit classifies the purchased products into groups that represent the order in which the purchased products will be placed into bags for carrying them home. The image acquisition unit acquires images of the purchased products stored in a basket or cart before and during the bagging process, and the identification unit identifies the bagged purchased products based on the images. The display control unit then displays the purchased products included in at least one of the classified groups on the display unit based on the bagged purchased products and the classification results.

[0073] This allows the customer to refer to the bagging group displayed on the display unit and bag the purchased items, eliminating the need for the customer to think about the order in which to bag the purchased items. As a result, the bagging support device can support the packing of purchased items.

[0074] In particular, the sorting unit sorts the purchased items into groups according to the order in which they will be packed. As a result, the bagging support device of this embodiment can present multiple candidates for purchased items that the customer should remove from the basket or cart. This allows the customer to select and remove items other than the purchased items that must be removed from the bottom of the basket or cart.

[0075] [Cases other than self-scan shopping] Here, the operation of the bagging support device 100 described in the embodiment will be described when a cash register other than that for self-scan shopping is used. Below, to the extent that the description of this embodiment is not unclear, explanations that overlap with the above description will be omitted. Cash registers other than those for self-scan shopping include cash registers where a store clerk processes everything from product registration to payment, semi-self-checkouts where a store clerk registers products and the customer processes payment, or full-self-checkouts where the customer processes everything from product registration to payment. In this case, the bagging support device 100 is a terminal device owned by the customer, or a terminal device installed on a workbench or checkout counter in the store.

[0076] At cash registers where a store clerk performs processes from product registration to payment, the clerk may also perform bagging. In this case, the bagging support device 100 may be included in a POS terminal used by the clerk as a cash register. Alternatively, the bagging support device 100 may be a terminal device installed near the cash register. Similarly, the display unit may be the display of a POS terminal used by the clerk as a cash register. Alternatively, the display unit may be the display of a terminal device installed near the cash register. Furthermore, when a store clerk uses bagging support, the bagging support device 100 performs operations in which the customer in the embodiment is replaced with the store clerk.

[0077] In cases where self-scan shopping is performed and a store clerk bagges the items at the register, the product information acquisition unit 101 may acquire information about the purchased items registered in the terminal device that is also the POS terminal device where the items were registered. Furthermore, in cases where the bagging assistance device is a terminal device separate from the POS terminal device where the items were registered, the product information acquisition unit 101 may acquire, for example, an identification number from a customer's point card or the like. The product information acquisition unit 101 may then acquire product information about the purchased items associated with the identification number. Note that the product information acquisition unit 101 may also acquire product information about purchased items associated with the identification number, including items purchased on the same day or items registered at the register within a predetermined time, such as one hour, as the purchased items to be bagged. Furthermore, for example, the product information acquisition unit 101 may read a receipt and acquire product information about purchased items associated with the identification number on the receipt or associated with a product identification number directly read from the receipt.

[0078] As in the embodiment, the image acquisition unit 103 acquires images of purchased products stored in a basket or cart before and during the bagging work, before being bagged. For example, the purchased products stored in a basket or cart before being bagged before being bagged may be photographed by a camera installed near the cash register where the products are registered.

[0079] [Modification 1] Next, Modification 1 of the present disclosure will be described in detail with reference to the drawings. Below, explanations of content that overlaps with the above explanation will be omitted to the extent that the explanation of this modification is not unclear.

[0080] 8 is a block diagram showing the configuration of a bagging support device according to Modification 1 of the present disclosure. A bagging support device 200 according to Modification 1 includes a bag information acquisition unit 206 in addition to the configuration of the embodiment.

[0081] The bag information acquisition unit 206 is an example of a bag information acquisition unit that acquires bag information. Here, the bag refers to the aforementioned take-out bag or the like. The bag information includes the bag size, the bag capacity, and the bag's load capacity. The bag's load capacity is the weight of contents that can be carried in the bag. The bag information may include the bag's material instead of the bag's load capacity. In this case, the bag's load capacity can be estimated from the load capacity per unit area of ​​the bag's material and the bag's size. The bag information acquisition unit 206 acquires, for example, bag information stored in association with the product identification number of the purchased bag included with the purchased item. Alternatively, for example, the bag information acquisition unit 206 may estimate bag information based on an image captured by the customer. Alternatively, for example, the bag information acquisition unit 206 may acquire information about the bag used by the customer that is pre-registered in association with the customer identification number of a point card or the like.

[0082] A customer may use one or more bags to place the purchased items in and carry them home. When the bag information acquisition unit 206 acquires information about one bag, the bag information acquisition unit 206 may determine whether the purchased items can be placed in one bag based on the bag information and the product information about the purchased items. For example, the bag information acquisition unit 206 may determine that the purchased items can be placed in one bag if the size of the bag is larger than the total size of the purchased items. Alternatively, the bag information acquisition unit 206 may determine that the purchased items can be placed in one bag if the bag's load capacity is larger than the total weight of the purchased items. When the bag information acquisition unit 206 determines that the purchased items cannot be placed in one bag, it may acquire further bag information. For example, when the bag information acquisition unit 206 determines that the purchased items cannot be placed in one bag, it may prompt the customer to take a photo of the bag or the bag's barcode.

[0083] If the bag information acquisition unit 206 acquires information on multiple bags, i.e., if there are multiple bags to put the purchased items in, the sorting unit 102 sorts the purchased items into groups for each bag according to the order in which they will be put in the bag. For example, the sorting unit 102 sorts the purchased items so that the weights are equal for each bag based on the product information. The sorting unit 102 may sort the purchased items into groups according to the order in which they will be put in the bag, and then sort the purchased items to be put in each of the multiple bags. Alternatively, the sorting unit 102 may sort the purchased items into each of the multiple bags, and then sort each bag into groups according to the order in which they will be put in the bag.

[0084] Furthermore, for example, the sorting unit 102 may classify the purchased items to be placed in each of the multiple bags by product attribute based on the product information. An example of a product attribute is the product temperature. When there are multiple bags, the sorting unit 102 sorts hot products and cold products into separate bags. In this case, room temperature products may be classified into either a bag containing hot products or a bag containing cold products. Furthermore, for example, an attribute of a product is a product category. When there are multiple bags, the sorting unit 102 may sort daily necessities and groceries into separate bags. Furthermore, the sorting unit 102 may sort only purchased items that are fragile or easily crushed into a smaller bag among the multiple bags. In this case, if there are multiple remaining bags, the sorting unit 102 sorts the other purchased items into the respective bags. Furthermore, the sorting unit 102 sorts each bag into a group according to the order in which the items will be placed in the bag.

[0085] Furthermore, when there are multiple bags for the purchased items, the display control unit 105 may cause the display unit to display a group of purchased items to be bagged first and the bag into which the purchased items will be bagged first. For example, the display control unit 105 may cause the display unit to display a group of purchased items to be bagged first and the bag into which the purchased items will be bagged first, such as "Daily necessities will be bagged first." The bags may be indicated by text information indicating characteristics such as color, material, shape, and size, a photographed image, a name set in advance by the customer, or the like. Furthermore, information indicating the bags may be acquired by the bag information acquisition unit 206 as bag information.

[0086] The bagging support device in this modified example described above can support bagging even when multiple bags are used.

[0087] [Modification 2] Next, Modification 2 of the present disclosure will be described in detail with reference to the drawings. Below, explanations of content that overlaps with the above explanation will be omitted to the extent that the explanation of this modification is not unclear.

[0088] 9 is a block diagram showing the configuration of a bagging support device according to Modification 2 of the present disclosure. A bagging support device 300 according to Modification 2 includes a determination unit 307 in addition to the configuration of the embodiment.

[0089] The determination unit 307 is an example of a determination unit for determining whether to provide bagging assistance. When a customer purchases only one, two, or three items, or when a customer purchases many identical items, the bagging assistance may not require a bagging order. In such cases, the determination unit 307 determines to provide bagging assistance, for example, when the number of purchased items is equal to or exceeds a predetermined number or the number of types of purchased items is equal to or exceeds a predetermined number, based on the product information. The predetermined number may be preset to a single value or a value for each customer. The predetermined number is, for example, an integer value such as 4 or 5. When determining whether to provide bagging assistance in this manner, the determination unit 307 determines whether to provide bagging assistance after the product information acquisition unit 101 acquires product information about the purchased items. If the determination is made to provide bagging assistance, the bagging assistance device 300 proceeds to the operations subsequent to the acquisition of product information in the bagging assistance described in the embodiment.

[0090] Furthermore, for example, the decision unit 307 may decide whether or not to provide bagging support based on input from the customer. For example, the decision unit 307 displays two buttons, one for providing bagging support and the other for providing bagging support, on the touch panel display of the terminal device. Then, the decision unit 307 decides whether or not to provide bagging support based on input from the customer on the touch panel display. When deciding whether or not to provide bagging support in this manner, the decision unit 307 displays, as an initial screen, a screen on the display unit that prompts the customer to input whether or not to provide bagging support. Then, when it is decided to provide bagging support, the bagging support device 300 proceeds to the bagging support operation described in the embodiment.

[0091] When the decision unit 307 decides not to provide bagging assistance, the display control unit 105 may, for example, return the display on the display unit to an initial screen. The initial screen is a screen that shows an operation to start the operation of the bagging assistance device, such as "touch the screen to start."

[0092] The bagging support device described above can provide bagging support according to the need for bagging support, thereby preventing customers from finding the bagging support bothersome.

[0093] [Hardware Configuration] Some or all of the components of each device in each embodiment and each modified example of the present disclosure described above are realized by any combination of an information processing device 1000 and a program, for example, as shown in Fig. 10. The information processing device 1000 includes, as an example, the following configuration.

[0094] - CPU (Central Processing Unit) 1001 - ROM (Read Only Memory) 1002 - RAM (Random Access Memory) 1003 - Program 1004 loaded into RAM 1003 - Storage device 1005 for storing program 1004 - Drive device 1007 for reading and writing from and to recording medium 1006 - Communication I / F 1008 for connecting to communication network 1009 - Input / output I / F 1010 for inputting and outputting data - Bus 1011 for connecting each component. Note that I / F is an abbreviation for Interface.

[0095] Each component of each device or system in each embodiment is realized by the CPU 1001 acquiring and executing a program that realizes the function of that component. The program that realizes the function of each component of each device is stored in the storage device 1005 or RAM 1003 in advance, for example, and is read by the CPU 1001 as needed. The program 1004 may be supplied to the CPU 1001 via a communication network, or may be stored in the recording medium 1006 in advance, and the drive device 1007 may read the program and supply it to the CPU 1001.

[0096] There are various variations in the method of realizing each device. For example, each device may be realized by any combination of a separate information processing device 1000 and a program for each component. Furthermore, multiple components included in each device may be realized by any combination of a single information processing device 1000 and a program.

[0097] Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits (circuitry) including a processor or the like, or a combination thereof. Examples of the circuits include a CPU, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and an LSI (Large Scale Integration). Examples of the LSI include an LSI dedicated to AI (Artificial Intelligence) processing. These may be configured by a single chip or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-described circuits and a program.

[0098] When some or all of the components of each device are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each device is connected via a communication network.

[0099] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0100] Furthermore, although the operations are described in a sequential order in the form of a flowchart, the order of the description does not limit the order in which the operations are performed. Therefore, when implementing each embodiment, the order of the operations may be changed as long as it does not affect the content.

[0101] This application claims priority based on Japanese Patent Application No. 2022-152676, filed September 26, 2022, the disclosure of which is incorporated herein in its entirety.

[0102] REFERENCE SIGNS LIST 100 Bagging support device 101 Product information acquisition unit 102 Classification unit 103 Image acquisition unit 104 Identification unit 105 Display control unit 200 Bagging support device 206 Bag information acquisition unit 300 Bagging support device 307 Determination unit 1000 Information processing device 1001 CPU 1002 ROM 1003 RAM 1004 Program 1005 Storage device 1006 Recording medium 1007 Drive device 1008 Communication I / F 1009 Communication network 1010 Input / output I / F 1011 Bus

Claims

1. A product information acquisition means for acquiring product information of a product purchased by a customer; a classification means for classifying the purchased products into groups representing the order in which the purchased products are to be put into bags for carrying the purchased products home, based on the product information; an image acquisition means for acquiring images of the purchased products contained in a basket or cart before and during the bagging operation; A means for identifying the packaged purchased items based on the image; a display control means for displaying on a display unit the purchased products included in at least one of the classified groups based on the packaged purchased products and the result of the classification; A bagging assistance device comprising:

2. The display control means displays a guide to temporarily place purchased items that are placed at the upper part of the basket or cart included in the image captured during the bagging work and are included in a group of purchased items to be bagged later than the current one.

2. The bagging assist device according to claim 1.

3. The identification means compares two of the images taken in chronological order, and identifies a purchased item that is included in the image taken earlier and not included in the image taken later as the packed purchased item.

3. The bagging assist device according to claim 2.

4. The display control means changes the display of the packaged purchased items to indicate that the packaged purchased items have been packaged.

2. The bagging assist device according to claim 1.

5. The identification means identifies a group as a group in which bagging has been completed when all purchased products included in the group are identified as the purchased products that have been bagged.

2. The bagging assist device according to claim 1.

6. The display control means changes the display of the group for which the packing has been completed to indicate that packing of the group has been completed.

6. The bagging assist device according to claim 5.

7. When there are a plurality of bags for putting the purchased products in, the sorting means sorts the purchased products for each bag into groups according to the order in which the purchased products are put into the bag.

7. A bagging assist device according to claim 1.

8. When there are a plurality of bags for the purchased products, the display control means displays on the display unit a group of the purchased products to be packed first and the bag to be packed first.

8. The bagging assist device according to claim 7.

9. The computer Obtain product information for the customer's purchase, classifying the purchased items into groups representing the order in which the purchased items are to be packed into bags for carrying the purchased items home based on the item information; Before and during the bagging operation, images are obtained of the purchased products contained in a basket or cart before being bagged; Identifying the packaged purchased items based on the image; A display unit displays the purchased products included in at least one of the classified groups based on the bagged purchased products and the result of the classification. How to help pack bags.

10. Obtain product information for the customer's purchase, classifying the purchased items into groups representing the order in which the purchased items are to be packed into bags for carrying the purchased items home based on the item information; Before and during the bagging operation, images are obtained of the purchased products contained in a basket or cart before being bagged; Identifying the packaged purchased items based on the image; A display unit displays the purchased products included in at least one of the classified groups based on the bagged purchased products and the result of the classification. A program that causes a computer to carry out processing.