Information processing system, control method, and program

Abstract

To provide technology to accurately determine the validity of articles recognized by a device that registers articles to be settled.SOLUTION: An information processing device having a function for registering an article to be settled includes: first acquisition means for acquiring article information relating to the article to be settled; second acquisition means for acquiring image information including the article; and determination means for determining, based on the output of the first and second acquisition means, whether or not there is any irregularity with respect to the article registration. The image information includes at least one of an image of the article when the article is placed on the registered article placement site or an image of the article placed on the registered article placement site.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] ,

[0001] The present invention relates to a technique for determining whether a product recognized by a device is the correct product.

Background Art

[0002] In a device for registering products to be settled, when a product to be settled is recognized using a code for identifying the product, such as a so-called barcode, a weight sensor is provided at a place where the recognized product is temporarily placed to determine the validity of the product recognized by the device. Here, "valid" means that in a device for registering products to be settled, the correct product is recognized as the product purchased by the customer. In this technique, based on the weight measured by the weight sensor and the weight of the product associated with the product identification code, it is determined whether the two match.

[0003] Another example of a technique for appropriately determining the validity of a product recognized by a device for registering products to be settled is described in, for example, Patent Document 1. Patent Document 1 discloses a technique for obtaining an image of an object (product) to which a barcode is attached and a basic shape of a product associated with the barcode in response to reading of the barcode for identifying the product, and determining the validity of the recognized product by a correlation value calculated using the shape of the object recognized from the image and the basic shape associated with the read barcode.

[0004] Patent documents 2 and 3 disclose technologies for identifying products through image-based object recognition. Specifically, patent documents 2 and 3 disclose technologies in which a product image is linked to a product ID, and the product image is matched with objects contained in the captured image based on feature quantities such as surface color, pattern, surface condition (including surface irregularities), and shape, and candidate products corresponding to the objects in the image are identified based on the degree of similarity. Furthermore, patent document 3 discloses a technology in which a thermal camera using far-infrared light is used to detect a region corresponding to a person's hand in an image, and the aforementioned feature quantity matching is performed after excluding that region. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2014-132501 [Patent Document 2] Japanese Patent Publication No. 2013-182323 [Patent Document 3] Japanese Patent Publication No. 2014-052806 [Overview of the project] [Problems that the invention aims to solve]

[0006] As described above, the accuracy of the technology for determining the validity of products recognized by the device that registers the products to be settled is preferable as much as possible.

[0007] The object of the present invention is to provide a technology for accurately determining the validity of goods recognized by a device that registers goods subject to settlement. [Means for solving the problem]

[0008] According to the present invention, A first acquisition means is provided in a container for storing and transporting goods, for acquiring product information relating to the goods subject to settlement, The container is provided with a second acquisition means for acquiring image information of the product that the customer places inside the container, An output means that outputs a warning when it is determined that the product placed in the container by the customer is fraudulent, using the information obtained by the first and second acquisition means, Equipped with, The first acquisition means differs from the second acquisition means. An information processing system is provided.

[0009] According to the present invention, A computer installed in a container for storing and transporting goods, The first acquisition step involves obtaining product information about the products subject to settlement, A second acquisition step involves acquiring image information of the product that the customer will place in the container, An output step that uses the information obtained in the first and second acquisition steps to output a warning if it is determined that the product placed in the container by the customer is fraudulent, Execute, The first acquisition step is provided with a control method different from that of the second acquisition step.

[0010] According to the present invention, A computer installed in a container for storing and transporting goods, A first means of obtaining product information regarding the products subject to settlement, A second acquisition means for acquiring image information of the product that the customer places in the container, An output means that outputs a warning when it is determined that the product placed in the container by the customer is fraudulent, using the information obtained by the first and second acquisition means, To make it function as, The first acquisition means is provided with a program different from that of the second acquisition means. [Effects of the Invention]

[0011] According to the present invention, the validity of products recognized by a device that registers products subject to settlement can be determined with high accuracy.

Brief Description of Drawings

[0012] [Figure 1] It is a diagram conceptually showing the processing configuration of the information processing apparatus in the first embodiment. [Figure 2] It is a diagram showing an example of the information stored in the size / shape information storage unit. [Figure 3] It is a diagram conceptually showing the hardware configuration of the information processing apparatus. [Figure 4] It is a flowchart showing the processing flow of the information processing apparatus in the first embodiment. [Figure 5] It is a diagram showing an example of the attachment of the information processing apparatus in the second embodiment. [Figure 6] It is a diagram conceptually showing the processing configuration of the information processing apparatus in the third embodiment. [Figure 7] It is a flowchart showing the processing flow of the information processing apparatus in the third embodiment. [Figure 8] It is a diagram conceptually showing the processing configuration of the information processing apparatus in the fourth embodiment. [Figure 9] It is a diagram showing an example of the information stored in the weight information storage unit. [Figure 10] It is a flowchart showing the processing flow of the information processing apparatus in the fourth embodiment. [Figure 11] It is a diagram conceptually showing the processing configuration of the information processing apparatus in the fifth embodiment. [Figure 12] It is a flowchart showing the processing flow of the information processing apparatus in the fifth embodiment. [Figure 13] It is a diagram conceptually showing the system configuration of the settlement system in the sixth embodiment. [Figure 14] It is a sequence diagram showing the processing flow of the settlement system in the sixth embodiment.

Modes for Carrying Out the Invention

[0013] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate.

[0014] [First Embodiment] [Processing Configuration] Figure 1 is a conceptual diagram showing the processing configuration of the information processing device 10 in the first embodiment. The information processing device 10 is a product registration device such as a so-called POS (Point of Sales) terminal that has at least a function to register products to be settled. The information processing device 10 may further have a function to settle the price of the registered products. The information processing device 10 in this embodiment includes an image recognition unit 110, a product recognition unit 120 (code acquisition unit 122, imaging unit 124, and thermography information acquisition unit 126), a size / shape information acquisition unit 130, a size / shape information storage unit 132, and a determination unit 140.

[0015] <Registering items to be settled> In this embodiment, the products subject to settlement are registered as follows.

[0016] Products subject to settlement are moved as shown by the dotted line in Figure 1. That is, products that have not yet been registered as products subject to settlement (hereinafter referred to as "unregistered products") are removed from the temporary storage location for unregistered products and moved to the location of the product recognition unit 120. Here, as will be described in detail later, the unregistered products are registered as products subject to settlement. Then, products that have been registered as products subject to settlement (hereinafter referred to as "registered products") are moved to the temporary storage location for registered products.

[0017] Unregistered products are registered as products eligible for settlement, for example, as follows: First, the code acquisition unit 122 of the product recognition unit 120 acquires the product identification code of the unregistered product held over the product recognition unit 120 as the product identification code of the product eligible for settlement. Here, the product identification code is data formed from letters, numbers, symbols, etc., predetermined in order to identify each product. Product identification codes may include PLU (Price Look Up) codes (for example, JAN (Japan Article Number) codes) or Non-PLU codes.

[0018] The product identification code is obtained by detecting and analyzing the product information symbol from the product being paid for. The product information symbol is a symbol that identifies information about a product. Here, the symbol is a barcode, a two-dimensional code (such as a QR code®), or a string symbol. Note that the string here also includes a sequence of numbers. The product information symbol is a barcode or the like that encodes information that identifies the product (such as a product information ID), or a string symbol that represents information that identifies the product. In this case, for example, the product identification code can be obtained from the product information symbol by using a code reader equipped with a light source and a photodetector such as a photodiode, or a code reader equipped with a lens and an image sensor such as a CCD (Charge Coupled Device) image sensor. In the case of a code reader equipped with a lens and an image sensor, the product identification code can also be obtained by extracting the product's features (e.g., color, shape, packaging, or specific marks or strings that identify the product) from an image containing the product being paid for, and performing a matching process by referring to a storage unit (not shown) that stores the product identification code in association with the product's features.

[0019] The product recognition unit 120 then uses the product identification code obtained by the code acquisition unit 122 to refer to a storage unit (not shown) that stores the product identification code and product information (for example, product name, unit price, whether or not there are discounts or price reductions), and reads the product information linked to the acquired product identification code. The product recognition unit 120 then adds the read product information as information to be used in the settlement process executed by the information processing device 10. As a result, the product before registration is recognized on the information processing device 10 and registered as a product subject to settlement.

[0020] <Determination of the validity of products recognized by the device> Furthermore, in this embodiment, the validity of the products registered as items to be settled as described above is determined by the processing units described below. In this embodiment, "valid" means that the information processing device 10 recognizes the correct products as items to be purchased by the customer.

[0021] The imaging unit 124 is a so-called 3D camera, and is configured to generate images by associating them with depth information indicating the distance in the depth direction of products (unregistered products) included in the imaging range. The imaging unit 124 generates images by associating them with depth information using known methods, such as the method using parallax in a stereo camera. In this embodiment, the imaging unit 124 includes the vicinity of the product recognition unit 120 in its imaging range, and its orientation and position are adjusted so that it can image unregistered products that have been moved to the product recognition unit 120. The imaging unit 124 may be provided integrally with the information processing device 10, such as a camera used when recognizing products as objects, or it may be installed around the information processing device 10, such as an external camera.

[0022] Furthermore, the imaging unit 124 preferably captures an image in accordance with the timing at which the code acquisition unit 122 reads the product identification code. For example, if the code acquisition unit 122 detects the presence of a product information symbol, it sends a notification to the imaging unit 124 indicating this. Upon receiving this notification, the imaging unit 124 can know the timing at which the code acquisition unit 122 reads the code. In this way, for products that have the aforementioned product information symbols, if the imaging unit 124 captures an image at the same time that the code acquisition unit 122 reads the product identification code, the orientation of the product included in the image will be stable, and the change in orientation from image to image will be reduced. This is expected to reduce the processing load by reducing the number of templates used for image matching, and to improve the recognition accuracy through image matching.

[0023] The thermographic information acquisition unit 126 is a so-called thermographic camera that uses infrared light, etc. It captures the infrared radiation energy emitted by objects included in the imaging range and generates thermographic information showing the temperature distribution of those objects. The thermographic information acquisition unit 126 is configured to acquire thermographic information in accordance with the imaging timing of the imaging unit 124. Furthermore, the position and orientation of the thermographic information acquisition unit 126 are adjusted so that it can acquire thermographic information for the same imaging range as the imaging range of the imaging unit 124 or for an imaging range that encompasses the imaging range of 124. The thermographic information acquisition unit 126 then aligns the positional relationship between the image generated by the imaging unit 124 and the thermographic information, for example, by performing coordinate transformation processing in the image space. In this way, the thermographic information acquisition unit 126 acquires thermographic information associated with the image generated by the imaging unit 124.

[0024] The image recognition unit 110 performs the following processing using the image generated by the imaging unit 124 and the thermographic information acquired by the thermographic information acquisition unit 126.

[0025] The image recognition unit 110 applies known image recognition processing to the image generated by the imaging unit 124 to detect, for example, the edges of products in the image. The image recognition unit 110 then matches the shape indicated by the detected edges with pre-set 2D or 3D shape template data and recognizes the shape with the highest degree of similarity as the shape of the product. Furthermore, the image recognition unit 110 uses the depth information associated with the image generated by the imaging unit 124 and the size of the area occupied by the product in the image to calculate, for example, the actual size of the product in the image (e.g., height, width, depth width, area of ​​each surface, etc.). Since the actual distance from the imaging device to the product can be determined based on the depth information, the image recognition unit 110 can convert the size of the area occupied by the product in the image to the actual size of the product based on that distance.

[0026] Furthermore, the image recognition unit 110 uses the thermographic information acquired by the thermographic information acquisition unit 126 to exclude areas estimated to be people from the shape of the product recognized from the image generated by the imaging unit 124. For example, the image recognition unit 110 has information indicating the range of a person's body temperature (e.g., 35-40°C) pre-set, and the image recognition unit 110 extracts areas indicating the set temperature from the thermographic information. Then, based on the positional relationship between the thermographic information acquired by the thermographic information acquisition unit 126 and the image generated by the imaging unit 124, the image recognition unit 110 excludes the area corresponding to the extracted area from the target area for edge detection processing, or excludes the area corresponding to the extracted area from the shape recognized based on the detected edges. As a result, the image recognition unit 110 can recognize a more accurate area corresponding to a product from the image generated by the imaging unit 124, and as a result, the actual size of the product converted from that area can be calculated with greater accuracy.

[0027] The size / shape information acquisition unit 130 uses the product identification code acquired by the code acquisition unit 122 to refer to the size / shape information storage unit 132 and acquire the size / shape information associated with the product identification code. Size / shape information refers to information indicating the size of each product (for example, the height, width, depth of the product, and the area of ​​each surface calculated from these) and the external shape of each product. The size / shape information storage unit 132 stores the product identification code and the size / shape information in association, as shown in Figure 2, for example. Figure 2 is a diagram showing an example of the information stored by the size / shape information storage unit 132. The information stored by the size / shape information storage unit 132 is not limited to the example in Figure 2.

[0028] The size / shape information acquisition unit 130 searches the size / shape information storage unit 132 to detect a product identification code identical to the product identification code acquired by the code acquisition unit 122, and acquires the size / shape information associated with the detected product identification code. In the example in Figure 1, the size / shape information storage unit 132 is shown to be provided in the information processing device 10, but the size / shape information storage unit 132 may be provided in another device that can communicate with the information processing device 10. In this case, the size / shape information acquisition unit 130 acquires the size / shape information associated with the product identification code acquired by the code acquisition unit 122 by communicating with the other device.

[0029] The determination unit 140 determines whether the size or shape recognized from the image generated by the imaging unit 124 and the size / shape information obtained using the product identification code meet predetermined criteria. Here, the predetermined criteria are criteria related to the size or shape of the product, and are criteria that allow confirmation that the product recognized from the image and the product identified from the product identification code are the same product. For example, the determination unit 140 calculates the similarity between the two according to a predetermined similarity calculation algorithm related to size and shape, and determines whether the similarity is equal to or greater than a predetermined standard value (a threshold value set in advance as the value for which products are considered to be the same product).

[0030] If the degree of agreement between the two is above a predetermined threshold, the determination unit 140 can determine that the product moved to the product recognition unit 120 and the product recognized by the information processing device 10 are the same product, and the determination unit 140 can determine that the recognition result by the information processing device 10 is valid. On the other hand, if the degree of agreement between the two is below a predetermined threshold, the determination unit 140 can determine that the product moved to the product recognition unit 120 and the product recognized by the information processing device 10 are different products, and the determination unit 140 can determine that the recognition result by the information processing device 10 is invalid. In this case, it is likely that some kind of defect has occurred, such as a code reading error or the application of an incorrect code. Therefore, if the degree of agreement between the two is below a predetermined threshold, the determination unit 140 may be configured to display a message prompting the product to be re-read or a predetermined warning message on the display unit of the information processing device 10, or to output it as an audible message.

[0031] In this embodiment, the size / shape information storage unit 132 may store multiple size / shape information associated with each product identification code. Specifically, the size / shape information storage unit 132 may store size / shape information indicating the orientation of the product as viewed from multiple directions, such as size / shape information as viewed from the front, size / shape information as viewed from above, and size / shape information as viewed from diagonally above, associated with each product identification code. In this case, the determination unit 140 calculates the degree of similarity between each of the multiple size / shape information associated with each product identification code and the size or shape recognized by the image recognition unit 110, and determines whether the degree of agreement between the two is equal to or greater than a predetermined standard value. By doing so, it is possible to suppress a decrease in the accuracy of the determination by the determination unit 140 due to the orientation of the product included in the image generated by the imaging unit 124.

[0032] [Hardware configuration] Figure 3 is a conceptual diagram showing the hardware configuration of the information processing device 10. As shown in Figure 3, the information processing device 10 includes a CPU 101, memory 102, storage 103, input / output interface (input / output I / F) 104, and a communication module 105, among others. The CPU 101, memory 102, storage 103, input / output interface 104, and communication module 105 are connected to each other by a data transmission path for sending and receiving data.

[0033] Memory 102 is a type of memory such as RAM (Random Access Memory) or ROM (Read Only Memory). Storage 103 is a storage device such as a hard disk, SSD (Solid State Drive), or memory card. Storage 103 stores program modules for realizing the functions of each processing unit of the information processing device 10, including the image recognition unit 110, product recognition unit 120, code acquisition unit 122, imaging unit 124, thermography information acquisition unit 126, size / shape information acquisition unit 130, and determination unit 140. The CPU 101 realizes the functions of each processing unit by executing these program modules. When executing the above modules, the CPU 101 may read these modules into memory 102 before execution, or it may execute them without reading them into memory 102. Storage 103 also functions as a size / shape information storage unit 132.

[0034] The input / output interface 104 is connected to a display device 1041, an input device 1042, an imaging device 1043, and a reading device 1044, etc. The display device 1041 is a device that displays a screen corresponding to drawing data processed by a CPU 101, a GPU (Graphics Processing Unit) (not shown), etc., such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. One or more display devices 1041 may be connected to the input / output interface 104. The input device 1042 is a device that receives input from the operator's operation, and is implemented as, for example, a keyboard, mouse, touch sensor, etc. The display device 1041 and the input device 1042 may be integrated and implemented as a touch panel. The imaging device 1043 is a so-called 3D camera, etc., and includes a monocular imaging module or a bicular imaging module (not shown). In this case, the imaging device 1043 can realize the functions of the imaging unit 124 described above. Furthermore, the imaging device 1043 is a so-called thermographic camera, and includes an imaging module (not shown) that detects infrared energy emitted from an object and generates an image showing the temperature distribution of the object based on the amount of infrared energy. In this case, the imaging device 1043 can realize the functions of the thermographic information acquisition unit 126 described above. The reading device 1044 is a device such as a code reader equipped with a light source and a light-receiving element such as a photodiode, or a code reader equipped with a lens and an image sensor such as a CCD (Charge Coupled Device) image sensor. The reading device 1044 can realize the functions of the code acquisition unit 122 described above.

[0035] The communication module 105 is used to send and receive data with external devices. There are various ways to connect the information processing device 10 to external devices via the communication module 105. For example, this connection can be via a bus line (e.g., a USB (Universal Serial Bus) line) or a network line. The network line may be wireless or wired.

[0036] Note that the hardware configuration of the information processing device 10 is not limited to the configuration shown in Figure 3. For example, various devices necessary for the settlement process of goods (such as a money input device and a change dispensing device) may be connected to the input / output interface 104.

[0037] [Example of operation] An example of the operation of the information processing device 10 of this embodiment will be explained using Figure 4. Figure 4 is a flowchart showing the processing flow of the information processing device 10 of the first embodiment.

[0038] When a product is moved to the location of the product recognition unit 120, the code acquisition unit 122 acquires the product identification code of that product (S101). In response to the acquisition of the product identification code, the imaging unit 124 generates an image of the product by associating it with depth information indicating the distance in the depth direction (S102), and the thermographic information acquisition unit 126 acquires thermographic information by associating it with the image generated by the imaging unit 124 (S103).

[0039] The image recognition unit 110 then uses the generated image and the depth information and thermographic information associated with the image to recognize the size or shape of the product (S104). For example, the image recognition unit 110 uses the actual distance to the product obtained from the depth information to convert the size of the area occupied by the product in the image into the actual size of the product. Alternatively, the image recognition unit 110 may use known image recognition processing, such as edge processing, to extract the area of ​​the product from the image and obtain the shape of that area as the shape of the product.

[0040] Furthermore, the size / shape information acquisition unit 130 refers to the size / shape information storage unit 132 using the product identification code acquired by the code acquisition unit 122, and reads the size / shape information associated with the product identification code from the size / shape information storage unit 132 (S105).

[0041] The determination unit 140 compares the size or shape of the product recognized by the image recognition unit 110 with the size / shape information associated with the product identification code and determines whether or not it meets a predetermined criterion (S106). For example, the determination unit 140 can calculate the similarity between the two according to a predetermined similarity calculation algorithm and determine whether or not the two meet a predetermined criterion based on whether or not the similarity is above a predetermined threshold. If the determination unit 140 determines that the predetermined criterion is met (S106: YES), it determines that the product has been successfully recognized by the information processing device 10 and does not execute the process in S107 described below. On the other hand, if the determination unit 140 determines that the predetermined criterion is not met (S106: NO), it outputs a predetermined output (for example, a message prompting the product to reload, a warning message, a warning sound, etc.) via a display or speaker (not shown) (S107).

[0042] [Operation and Effects of the First Embodiment] As described above, according to this embodiment, for a product read by the product recognition unit 120, the similarity between the actual size or shape of the product calculated from the image and the size / shape information indicating the size or shape of the product linked to the product identification code of the product is calculated. Then, whether the similarity between the two is equal to or greater than a predetermined threshold (a threshold set in advance as a value that will be considered to be the same product) determines whether the two meet a predetermined standard. If the size or shape of the two are approximately the same and the predetermined standard is met, there is a high probability that the product moved to the position of the product recognition unit 120 and the product recognized by the information processing device 10 are the same, and the determination unit 140 determines that the recognition result by the information processing device 10 is valid. On the other hand, if the size or shape of the two do not match and the predetermined standard is not met, there is a high probability that the product moved to the position of the product recognition unit 120 and the product recognized by the information processing device 10 are different, and the determination unit 140 determines that the recognition result by the information processing device 10 is invalid. In this way, by comparing the actual size or shape of the product recognized from the image with the theoretical size or shape of the product linked to the product identification code, the validity of the product recognized by the information processing device 10 can be determined with high accuracy.

[0043] [Second Embodiment] In one operational method, a device for reading items to be paid for is attached to the basket in which customers carry their purchased goods. When an item is placed in the basket, it is registered as an item to be paid for, and the cash register terminal only processes the payment. In such a case, by equipping the basket with a device having the same configuration as the information processing device 10 of the first embodiment, the same effects as the first embodiment can be obtained.

[0044] [Processing configuration / Hardware configuration] The information processing device 10 of this embodiment includes an image recognition unit 110, a product recognition unit 120 (code acquisition unit 122, imaging unit 124, thermography information acquisition unit 126), a size / shape information acquisition unit 130, a size / shape information storage unit 132, and a determination unit 140, similar to the configuration shown in Figure 1. Furthermore, the information processing device 10 of this embodiment also has the same hardware configuration as the first embodiment, and each processing unit is implemented in the same manner as the first embodiment. However, each hardware component of the information processing device 10 shown in Figure 4 is smaller than that of the first embodiment in order to be mounted on the basket 20.

[0045] Figure 5 shows an example of the installation of the information processing device 10 in the second embodiment. As shown in Figure 5, the information processing device 10 is attached to the basket 20. As shown in Figure 5, unregistered items are registered as items to be settled by the information processing device 10 and then placed in the basket 20.

[0046] [Example of operation] As shown in Figure 4, the information processing device 10 of this embodiment operates in the same manner as the information processing device 10 of the first embodiment.

[0047] [Operation and Effects of the Second Embodiment] As described above, the same actions and effects as those of the first embodiment can be obtained with this embodiment as well.

[0048] [Third Embodiment] This embodiment is the same as the first and second embodiments, except that the information processing device 10 further includes a count acquisition unit 150.

[0049] [Processing Configuration] Figure 6 is a conceptual diagram showing the processing configuration of the information processing device 10 in the third embodiment. As shown in Figure 6, the information processing device 10 in this embodiment further includes a count acquisition unit 150.

[0050] The count acquisition unit 150 acquires the number of products included in the imaging range of the image generated by the imaging unit 124. Specifically, the count acquisition unit 150 acquires the number of product shapes recognized by the image recognition unit 110 as the number of products. Alternatively, the count acquisition unit 150 acquires the number of objects detected within the image as the number of products by pattern matching of pre-stored product features.

[0051] In this embodiment, the determination unit 140 determines whether the number of product identification codes obtained by the code acquisition unit 122 matches the number of products obtained by the quantity acquisition unit 150. Here, "match" means an exact match. In this embodiment, if multiple product information symbols exist in the image, the code acquisition unit 122 detects each product information symbol by, for example, performing image recognition processing, and obtains multiple product identification codes at approximately the same time. The code acquisition unit 122 in this embodiment further notifies the determination unit 140 of the number of product identification codes obtained here.

[0052] [Hardware configuration] The information processing device 10 of this embodiment also has a hardware configuration as shown in Figure 4, similar to the first and second embodiments. The storage 103 further stores a program module that realizes the function of the count acquisition unit 150, and the count acquisition unit 150 is realized when the CPU 101 executes the program module.

[0053] [Example of operation] An example of the operation of the information processing device 10 of this embodiment will be explained using Figure 7. Figure 7 is a flowchart showing the processing flow of the information processing device 10 of the third embodiment. In the following, we will mainly explain the steps (S201 to S203) that differ from those of the first and second embodiments.

[0054] The code acquisition unit 122, when acquiring product identification codes, counts the number of acquired product identification codes (S201). The count acquisition unit 150 then acquires the number of product shapes recognized by the image recognition unit 110 as the number of products included in the image generated by the imaging unit 124 (S202). The determination unit 140 compares the number of product identification codes counted in S201 with the number of products acquired in S202 to check whether they match (S203). If the number of products and the number of product identification codes do not match (S203: NO), the determination unit 140 determines that some kind of error has occurred, such as a product reading error, and displays a predetermined message on the display or outputs it as audio information, such as a message prompting the product to be reread or a warning message (S107). On the other hand, if the number of items matches the number of item identification codes (S203: YES), the determination unit 140 determines that no error such as a misreading of the items has occurred, and as described in detail in the first embodiment, it checks whether the size or shape obtained from the image and the size / shape information obtained from the item identification code meet predetermined criteria (S106).

[0055] [Operation and Effects of the Third Embodiment] In this embodiment, it is further determined whether the number of products recognized from the image matches the number of product identification codes actually recognized by the information processing device 10. This makes it possible to determine whether an error such as a product reading error has occurred, and if such an error has occurred, it is possible to prompt the information processing device 10 to perform accurate product recognition.

[0056] [Fourth Embodiment] This embodiment describes a configuration in which the information processing device 10 described in each of the above embodiments is combined with a technology that uses a weight sensor.

[0057] When the validity of a product recognized by the information processing device 10 is determined based on the weight measured by a weight sensor and the weight of the product linked to the product identification code, the correct determination result may not be obtained if the weight of the product is extremely light or if there is variation in weight even for the same product. In the case of such products, the information processing device 10 of this embodiment determines the validity of the recognition based on the size or shape of the product recognized from the image and the size / shape information obtained from the product identification code, as described in each of the embodiments described above.

[0058] [Processing Configuration] Figure 8 is a conceptual diagram showing the processing configuration of the information processing device 10 in the fourth embodiment. In this embodiment, the information processing device 10 is equipped with weight sensors (160 and 162) in the locations where unregistered products and registered products are placed, respectively, and basically determines the validity of the products recognized by the information processing device 10 based on the amount of change in weight measured by these weight sensors.

[0059] As shown in Figure 8, the information processing device 10 of this embodiment further includes a weight information storage unit 142. The weight information storage unit 142 stores, linked to a product identification code, at least one of the following: information indicating the weight of one product with the product identification code (hereinafter referred to as weight information), or information indicating the weight deviation of the product (hereinafter referred to as weight deviation information). An example of the information stored by the weight information storage unit 142 is shown in Figure 9. Figure 9 is a diagram showing an example of the information stored by the weight information storage unit 142. In the example in Figure 9, the weight information storage unit 142 stores weight information and weight deviation information linked to a product identification code.

[0060] In this embodiment, the determination unit 140 obtains a product identification code from the code acquisition unit 122 and uses the product identification code to obtain weight information or weight deviation information associated with the product identification code obtained by the code acquisition unit 122 from the weight information storage unit 142. If the weight information storage unit 142 stores information as shown in Figure 9, the determination unit 140 can obtain both weight information and weight deviation information. Based on the obtained weight information or weight deviation information, the determination unit 140 determines whether at least one of the following conditions is met: (1) the weight associated with the acquired product identification code is equal to or greater than a standard value, (2) the weight associated with the acquired product identification code is not set, or (3) the weight deviation associated with the acquired product identification code is equal to or greater than a predetermined threshold.

[0061] These conditions determine whether the product whose product identification code is obtained by the code acquisition unit 122 is unsuitable for processing by the weight sensor. Condition (1) above determines whether a product is unsuitable for processing by the weight sensor, for example, if the product is very light and the amount of weight change that can be detected by the weight sensor is small. Condition (2) above determines whether a product is unsuitable for processing by the weight sensor if the weight is not set. Condition (3) above determines whether a product is unsuitable for processing by the weight sensor, for example, if the weight varies greatly.

[0062] If the determination unit 140 of this embodiment determines that any of (1) to (3) above applies, it executes the determination process described in each of the above embodiments, that is, the process of determining the validity of the recognition by the information processing device 10 using the size or shape of the product. Specifically, if the determination unit 140 of this embodiment determines that any of (1) to (3) above applies, it notifies the image recognition unit 110 and the size / shape information acquisition unit 130 of the execution request for the process described in each of the above embodiments. Upon receiving this notification, the image recognition unit 110 sends the size or shape of the product recognized from the image generated by the imaging unit 124 to the determination unit 140. The code acquisition unit 122, upon receiving this notification, sends the size / shape information associated with the product identification code acquired by the code acquisition unit 122 to the determination unit 140. The determination unit 140 uses the size or shape of the product received from the image recognition unit 110 and the size / shape information received from the size / shape information acquisition unit 130 to execute the determination process as described in each of the above embodiments.

[0063] [Hardware configuration] The information processing device 10 of this embodiment also has a hardware configuration as shown in Figure 4, similar to the first and second embodiments. The storage 103 further stores program modules that realize the functions of the determination unit 140 of this embodiment, and the determination unit 140 of this embodiment is realized when the CPU 101 executes these program modules. Furthermore, the storage 103 can realize the functions of the weight information storage unit 142.

[0064] [Example of operation] An example of the operation of the information processing device 10 of this embodiment will be explained using Figure 10. Figure 10 is a flowchart of the processing flow of the information processing device 10 of the fourth embodiment. In the following explanation, the flowchart in Figure 4 will be used as the basis, and the processes added in this embodiment (S301 to S303) will be mainly explained.

[0065] The determination unit 140 uses the product identification code acquired by the code acquisition unit 122 in S101 to refer to the weight information storage unit 142 and acquire weight information or weight deviation information associated with the product identification code (S301). Then, the determination unit 140 uses the acquired weight information or weight deviation information to determine whether the product with the product identification code acquired by the code acquisition unit 122 in S101 is unsuitable for determination processing using a weight sensor (S302). Specifically, the determination unit 140 determines whether at least one of the following conditions is met: (1) the weight associated with the acquired product identification code is equal to or greater than a standard value, (2) the weight associated with the acquired product identification code is not set, or (3) the weight deviation associated with the acquired product identification code is equal to or greater than a predetermined threshold.

[0066] If the product identification code acquired by the code acquisition unit 122 in S101 is not suitable for judgment processing using a weight sensor (S302: YES), the judgment unit 140 notifies the image recognition unit 110 and the size / shape information acquisition unit 130 of this fact, and the processing described in the first embodiment is executed (S102-S107). On the other hand, if the product identification code acquired by the code acquisition unit 122 in S101 is suitable for judgment processing using a weight sensor (S302: NO), the judgment processing using a weight sensor is executed (S303).

[0067] [Fourth Embodiment's Function and Effects] In this embodiment, the code acquisition unit 122 determines whether the product with the acquired product identification code is unsuitable for judgment processing using a weight sensor. If the code acquisition unit 122 determines that the product with the acquired product identification code is unsuitable for judgment processing using a weight sensor, a judgment process is executed based on the size or shape of the product recognized from the image and the size / shape information associated with the product identification code. This prevents misrecognition that may occur when using a weight sensor, etc., and improves the accuracy of determining the validity of the product recognized by the information processing device 10.

[0068] [Modification of the fourth embodiment] In this embodiment, the determination unit 140 basically performs determination processing based on the change in weight measured by the weight sensor, and for products that meet predetermined conditions, it performs determination processing based on the size or shape recognized from the image. However, the determination unit 140 is not limited to this, and may be configured to basically perform determination processing based on the size or shape recognized from the image, and for products that meet conditions different from the above predetermined conditions, it performs determination processing based on the change in weight measured by the weight sensor. Here, "products that meet different conditions" refers to products that are not suitable for image recognition processing, such as thin products that have no thickness, where many feature points may be recognized depending on the orientation and posture. In such cases, the information processing device 10 stores the product identification code of the product that is not suitable for image recognition processing in a storage unit (not shown), and when the code acquisition unit 122 acquires the same product identification code as the product identification code stored in the storage unit, the determination unit 140 determines the validity of the product recognized by the information processing device 10 based on the change in weight measured by the weight sensor.

[0069] [Fifth Embodiment] This embodiment describes an information processing device 10 having the minimum configuration necessary to achieve the effects of the present invention. It is provided as a separate device (for example, a server device) from the device for registering products to be settled (hereinafter referred to as the product registration device), and is capable of communicating with the product registration device via, for example, a LAN (Local Area Network).

[0070] [Processing Configuration] Figure 11 is a conceptual diagram showing the processing configuration of the information processing device 10 in the fifth embodiment. As shown in Figure 11, the information processing device 10 in this embodiment includes an image recognition unit 110, a code acquisition unit 122, a size / shape information acquisition unit 130, and a determination unit 140.

[0071] The image recognition unit 110 recognizes the size or shape of an item in an image using an image to which depth information indicating the distance of the item in the depth direction within the imaging range is associated. The code acquisition unit 122 and the size / shape information acquisition unit 130 acquire size / shape information associated with the acquired item identification code. The determination unit 140 determines whether the size or shape of the item recognized from the image and the size / shape information acquired using the item identification code meet predetermined criteria.

[0072] [Hardware configuration] The information processing device 10 of this embodiment has the hardware configuration shown in Figure 3. Note that the input / output interface 104 of the information processing device 10 of this embodiment is not connected to the imaging device 1043 and the reading device 1044, and may be configured to acquire images and product identification codes from other devices via the communication module 105. The image recognition unit 110, code acquisition unit 122, size / shape information acquisition unit 130, and determination unit 140 of this embodiment are realized by the CPU 101 executing program modules that realize the functions of each processing unit, similar to the embodiments described above.

[0073] [Example of operation] An example of the operation of the information processing device 10 of this embodiment will be explained using Figure 12. Figure 12 is a flowchart showing the processing flow of the information processing device 10 of the fifth embodiment.

[0074] The code acquisition unit 122 communicates with the product registration device via a network such as a LAN and acquires the product identification code of the product to be settled, which has been read by the product registration device (S401). Then, the size / shape information acquisition unit 130 uses the product identification code acquired in S401 to read the size / shape information associated with that product identification code (S402). In this embodiment, the size / shape information is stored in a memory unit (not shown) of another device that can communicate with the information processing device 10, for example, and is linked to the product identification code.

[0075] Furthermore, the image recognition unit 110 communicates with the product registration device via a network such as a LAN, and acquires an image generated by an imaging means provided in the product registration device (S403). This imaging means provided in the product registration device is a so-called 3D camera, which can generate images in association with depth information. The image recognition unit 110 then recognizes the shape of the product in the image using known image recognition processing, and calculates the actual size of the product based on the depth information associated with the image and the size of the product area in the image (S404).

[0076] The determination unit 140 determines whether the size / shape information acquired in S402 and the size and shape of the product acquired in S404 meet predetermined criteria (S405), and transmits the result to the product registration device (S406).

[0077] [Fifth Embodiment's Function and Effects] In this embodiment, the actual size or shape of the product recognized from the image is compared with the theoretical size or shape of the product associated with the product identification code. This allows for accurate determination of the validity of the product recognized by the product registration device.

[0078] [Sixth Embodiment] In this embodiment, a settlement system is described that includes a product registration device that performs the process of registering products to be settled, and a settlement device that performs the settlement process for the price of the products registered as products to be settled.

[0079] [System Configuration] Figure 13 is a conceptual diagram showing the system configuration of the settlement system 1 in the sixth embodiment. As shown in Figure 13, the settlement system 1 includes an information processing device 10 that functions as a product registration device, a settlement device 30 that performs settlement processing for the price of products registered by the information processing device 10, and a server device 40. In this case, the information processing device 10, the settlement device 30, and the server device 40 are connected to each other via a network such as a LAN. The information processing device 10 also has the same configuration as any one of the first to fourth embodiments. In the following description, an example will be given in which the information processing device 10 of this embodiment has the same configuration as in the first embodiment as shown in Figure 1. Note that, not limited to the example in Figure 13, the image recognition unit 110, the code acquisition unit 122, the size / shape information acquisition unit 130, and the determination unit 140 of the information processing device 10 may be provided as a separate device (for example, the server device 40) from the product registration device, and can communicate with the product registration device via, for example, a LAN, as described in the fifth embodiment. In this case, the product registration device has a processing unit that acquires a product identification code from a product information symbol or the like, in addition to the code acquisition unit 122 of the information processing device 10.

[0080] [Example of operation] The processing flow of the settlement system 1 of this embodiment will be explained using Figure 14. Figure 14 is a sequence diagram showing the processing flow of the settlement system 1 of the sixth embodiment.

[0081] As described in the first embodiment, the information processing device 10 executes the processes S101 to S105 in the flowchart of Figure 4. Then, it makes the determination at S106 in the flowchart of Figure 4. If the determination unit 140 determines that the predetermined criteria are not met (S106: NO), the determination unit 140 makes a predetermined output (for example, an output prompting the product to be recognized again) (S107), and the processes S101 to S105 are executed again. On the other hand, if the determination unit 140 determines that the predetermined criteria are met (S106: YES), the product recognition unit 120 determines that the product has been successfully recognized by the information processing device 10 and reads out the product information associated with the product identification code acquired by the code acquisition unit 122 (S501). Specifically, the product recognition unit 120 uses the product identification code acquired by the code acquisition unit 122 to refer to a storage unit (not shown) that stores product identification codes linked to product information (e.g., product name, unit price, whether or not there are discounts or price reductions), and reads the product information linked to the acquired product identification code. The product recognition unit 120 then adds the read product information as information used for settlement processing executed by the information processing device 10 (hereinafter referred to as settlement information) (S502). The settlement information includes at least the name of each product subject to settlement, the unit price and quantity of each product subject to settlement, and the total amount of all products subject to settlement. The processes S101-S107, S501, and S502 are repeated until the registration of products subject to settlement in one transaction is completed (S503:NO). The information processing device 10 detects events such as the pressing of a "subtotal" button by an input device provided on the information processing device 10, and detects that the registration of the products subject to settlement in a single transaction has been completed.

[0082] If it is detected that the registration of goods subject to settlement in a transaction has been completed (S503: YES), the information processing device 10 transmits the settlement information for that transaction, generated in S502, to the settlement device 30 via the server device 40 (S504). The information processing device 10 identifies the settlement device 30 to which the settlement information should be sent, for example, through an input operation of the information processing device 10 operator, and transmits the settlement information. In this case, the information processing device 10 transmits the settlement information and information specifying the settlement device 30 to which the settlement information should be sent to the server device 40. The server device 40 then transmits the settlement information to the settlement device 30 specified by the information processing device 10, and the settlement device 30 performs settlement processing using the settlement information transmitted from the server device 40 (S505). Furthermore, the information processing device 10 may, for example, transmit settlement information to a server device 40 and, along with providing the customer with a portable medium containing information identifying the settlement information and readable by a settlement device 30, and allow the customer to load the portable medium into any settlement device 30. In this case, the settlement device 30 reads the settlement information from the server device 40 using the identification information contained in the portable medium and performs the settlement process (S505).

[0083] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0084] For example, in the first embodiment, the imaging unit 124 may acquire an image when an unregistered product is removed from its storage location (e.g., a basket) in order to hold the product up to the product recognition unit 120 before registration, and the image recognition unit 110 may be configured to recognize the size or shape of the product from the image. In this case, it is possible to determine whether the unregistered product removed from the storage location matches the product that was actually registered as the item to be paid for. Alternatively, for example, the imaging unit 124 may acquire an image when a registered product is placed in its storage location, and the image recognition unit 110 may be configured to recognize the size or shape of the product from the image. In this case, it is possible to determine whether the registered product placed in the storage location matches the product that was actually registered as the item to be paid for. By doing this, it is possible to detect fraudulent activities such as product substitution during product registration and to determine whether the recognized product is the correct product.

[0085] Furthermore, while the flowcharts or sequence diagrams used in the above description show multiple processes in order, the execution order of the processes performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated processes can be changed to the extent that it does not impede the content. Also, the above embodiments can be combined to the extent that their contents do not conflict.

[0086] Examples of reference formats are provided below. 1. An image recognition means that recognizes the size or shape of an item in an image using an image to which depth information indicating the distance in the depth direction of the item included in the imaging range is associated, A code acquisition method for obtaining the product identification code of the product to be settled, A size / shape information acquisition means for acquiring size / shape information associated with the acquired product identification code, A determination means for determining whether the size or shape of the product recognized from the image and the size / shape information obtained using the product identification code meet predetermined criteria, A product registration device equipped with the following features. 2. The system further includes a means for acquiring thermographic information that acquires thermographic information in association with the aforementioned image, The aforementioned image recognition means is Using the acquired thermographic information, the portion estimated to be a person is excluded from the shape of the product recognized from the image. 1. The product registration device described in item 1. 3. Multiple pieces of size / shape information are associated with the aforementioned product identification code. The determination means determines whether the recognized size or shape and each of the plurality of size / shape pieces of information meet the predetermined criteria. A product registration device as described in 1. or 2. 4. The system further includes an imaging means for capturing the image in accordance with the timing at which the code acquisition means reads the product identification code. A product registration device as described in any one of items 1 through 3. 5. The system further includes a means for acquiring the number of products included in the imaging range, The determination means determines whether the number of acquired product identification codes matches the number of acquired products. A product registration device as described in any one of items 1 through 4. 6. The aforementioned product identification code is further linked to the weight or weight deviation of the product associated with that product identification code. The determination means performs the determination in at least one of the following cases: when the weight associated with the acquired product identification code is equal to or greater than a standard value; when the weight associated with the acquired product identification code is not set; or when the weight deviation associated with the acquired product identification code is equal to or greater than a predetermined threshold. A product registration device as described in any one of items 1 through 5. 7. Computers Using an image to which depth information indicating the distance in the depth direction of the products included in the imaging range is associated, the size or shape of the products in the image is recognized. Obtain the product identification code of the item to be settled, The size / shape information associated with the acquired product identification code is obtained, The difference between the size or shape of the product recognized from the image and the size / shape information obtained using the product identification code is calculated, and it is determined whether the difference meets a predetermined standard. A control method that includes the following. 8. The aforementioned computer, Thermographic information is acquired in association with the aforementioned image. Using the acquired thermographic information, the portion estimated to be a person is excluded from the shape of the product recognized from the image. The control method described in 7, including the above. 9. Multiple pieces of size / shape information are associated with the aforementioned product identification code. The aforementioned computer, The recognized size or shape and each of the plurality of size / shape pieces of information are used to determine whether or not they meet the predetermined criteria. The control method described in 7. or 8., which includes the following: 10. The aforementioned computer, Depending on the timing of reading the aforementioned product identification code, the image is captured. A control method as described in any one of paragraphs 7 to 9, including the following: 11. The aforementioned computer, The number of products included in the imaging range is obtained, The system determines whether the number of product identification codes obtained matches the number of products obtained. A control method according to any one of 7 to 10, including the following: 12. The aforementioned product identification code is further linked to the weight or weight deviation of the product associated with that product identification code. The aforementioned computer, The determination is performed in at least one of the following cases: when the weight associated with the acquired product identification code is equal to or greater than a standard value; when the weight associated with the acquired product identification code is not set; or when the weight deviation associated with the acquired product identification code is equal to or greater than a predetermined threshold. A control method according to any one of paragraphs 7 to 11, including the following: 13. Computers, Image recognition means that recognizes the size or shape of an item in an image using an image to which depth information indicating the distance of the item in the depth direction included in the imaging range is associated. A code acquisition method for obtaining the product identification code of the product to be settled. Size / shape information acquisition means for acquiring size / shape information associated with the acquired product identification code, A determination means that calculates the difference between the size or shape of the product recognized from the image and the size / shape information obtained using the product identification code, and determines whether the difference meets a predetermined standard. A program designed to function as such. 14. The aforementioned computer, Thermographic information acquisition means that acquires thermographic information in association with the aforementioned image, The image recognition means, A means for using the acquired thermographic information to exclude parts that are presumed to be human from the shape of the product recognized from the image, The program described in section 13 for further functionality. 15. Multiple pieces of size / shape information are associated with the aforementioned product identification code. The aforementioned computer, The determination means comprising a means for determining whether the recognized size or shape and each of the plurality of size / shape pieces of information satisfy the predetermined criteria, The program described in 13. or 14. for further functionality. 16. The aforementioned computer, In accordance with the timing at which the code acquisition means reads the product identification code, an imaging means captures the image. A program described in one of sections 13 through 15 to further enable this function. 17. The aforementioned computer, A means for obtaining the number of products included in the imaging range, The determination means comprising means for determining whether the number of acquired product identification codes matches the number of acquired products. A program described in one of sections 13 through 16 to further enable this function. 18. The aforementioned product identification code is further linked to the weight or weight deviation of the product associated with that product identification code. The aforementioned computer, The determination means, wherein the determination is performed in at least one of the following cases: when the weight associated with the acquired product identification code is equal to or greater than a standard value; when the weight associated with the acquired product identification code is not set; or when the weight deviation associated with the acquired product identification code is equal to or greater than a predetermined threshold. A program described in one of sections 13 through 17 to function as such. [Explanation of symbols]

[0087] 1. Settlement System 10 Information Processing Devices 101 CPU 102 memory 103 Storage 104 Input / Output Interfaces 1041 Display device 1042 Input device 1043 Imaging device 1044 Reader 105 Communication Module 110 Image Recognition Unit 120 Product recognition department 122 Code Acquisition Section 124 Imaging Unit 126 Thermographic Information Acquisition Unit 130 Size / Shape Information Acquisition Unit 132 Size / Shape Information Storage Unit 140 Judgment section 142 Weight information storage section 150 Quantity acquisition unit 170 Weight information acquisition section 20 Basketball 30 Payment device 40 Server Devices

Claims

1. A determination means that determines whether the goods placed in a container are improper, using the amount of change in weight of the container used to store and transport the goods, The output means outputs a warning if the determination means determines that the product placed in the container by the customer is inappropriate, Equipped with, The container is provided with a product information acquisition means for acquiring product information relating to the product to be settled, The container is provided with an image information acquisition means for acquiring image information of the product that the customer places inside the container, Furthermore, The output means uses the information obtained by the product information acquisition means and the image information acquisition means to output a warning if it is determined that the product placed in the container by the customer is inappropriate. The information acquired by the aforementioned product information acquisition means is further linked to the weight or weight deviation of the product. The determination means is an information processing system that determines the validity of a product placed in a container using information obtained by the product information acquisition means and the image information acquisition means in at least one of the following cases: when the weight associated with the information acquired by the product information acquisition means is less than or equal to a standard value; when the weight associated with the information acquired by the product information acquisition means is not set; or when the weight deviation associated with the information acquired by the product information acquisition means is greater than or equal to a predetermined threshold.

2. The warning includes a message prompting the product to reload, The information processing system according to claim 1.

3. The container is provided with a weight information acquisition means for acquiring the weight information of the product placed inside the container, The determination means uses the weight information to determine the validity of the product placed in the container. The information processing system according to claim 1 or 2.

4. A computer installed in a container for storing and transporting goods, A determination step of determining whether the product placed in the container is fraudulent using the amount of change in weight in the container, In the determination step, if it is determined that the product placed in the container by the customer is fraudulent, an output step is made to output a warning, Execute, Product information relating to the product subject to settlement is obtained from the product information acquisition means provided in the container, Image information of the product that the customer places in the container is acquired from the image information acquisition means provided in the container. The output step uses the information obtained by the product information acquisition means and the image information acquisition means to output a warning if it is determined that the product placed in the container by the customer is inappropriate. The information acquired by the aforementioned product information acquisition means is further linked to the weight or weight deviation of the product. The determination step is a control method that determines the validity of a product placed in a container using information obtained by the product information acquisition means and the image information acquisition means in at least one of the following cases: when the weight associated with the information acquired by the product information acquisition means is less than or equal to a standard value; when the weight associated with the information acquired by the product information acquisition means is not set; or when the weight deviation associated with the information acquired by the product information acquisition means is greater than or equal to a predetermined threshold.

5. A computer installed in a container for storing and transporting goods, A determination means for determining whether the product placed in the container is fraudulent using the amount of change in weight of the container, The output means outputs a warning if the determination means determines that the product placed in the container by the customer is inappropriate, To make it function as, The container is provided with a product information acquisition means for acquiring product information relating to the product to be settled, The container is provided with an image information acquisition means for acquiring image information of the product that the customer places inside the container, Furthermore, The output means uses the information obtained by the product information acquisition means and the image information acquisition means to output a warning if it is determined that the product placed in the container by the customer is inappropriate. The information acquired by the aforementioned product information acquisition means is further linked to the weight or weight deviation of the product. The determination means is a program that determines the validity of a product placed in the container using information obtained by the product information acquisition means and the image information acquisition means in at least one of the following cases: when the weight associated with the information acquired by the product information acquisition means is less than or equal to a standard value; when the weight associated with the information acquired by the product information acquisition means is not set; or when the weight deviation associated with the information acquired by the product information acquisition means is greater than or equal to a predetermined threshold.

Images