Information processing device and control method

Abstract

The goal is to solve the problems that arise when identifying products using multiple images generated by multiple imaging devices. [Solution] An information processing device comprising: an acquisition means for acquiring images of products to be paid for, captured by multiple imaging means; a recognition means for recognizing the products based on the images; and a notification means for notifying when the recognition result of the recognition means cannot be uniquely identified, wherein the recognition means determines the image to be used for recognition based on the position of the product to be recognized and the position or orientation of the imaging means.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a control method.

Background Art

[0002] In stores such as supermarkets, cash registers that are operated by an operator himself / herself have begun to be used. The operator registers the purchased goods by having the barcode or the like attached to the goods to be purchased recognized by the cash register. After that, the operator purchases the registered goods by inserting the price displayed on the screen into the cash register.

[0003] As a document that discloses a technique related to such a cash register, for example, there is Patent Document 1. Patent Document 1 has one scanner and one camera each, and discloses a technique for determining the identity of a scanned product and an imaged product by using the result of scanning the product and the result of imaging the product by the camera.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In order to improve the recognition accuracy of products, it is conceivable to identify products using a plurality of images generated by a plurality of imaging means.

[0006] The present invention has been made in view of the above problems. An object of the present invention is to solve the problems that occur when identifying a product using a plurality of images generated by a plurality of imaging means.

Means for Solving the Problems

[0007] The information processing apparatus of the present invention includes an acquisition means for acquiring images of products to be settled captured by a plurality of imaging means, and a recognition means for recognizing the products based on the images. The system includes a notification means that provides notification when the recognition result of the recognition means cannot be uniquely identified, The recognition means is an information processing device that determines an image to be used for recognition based on the position of the product to be recognized and the position or orientation of the imaging means.

[0008] Another information processing device of the present invention comprises an acquisition means for acquiring images of products to be settled, captured by a first imaging means, a second imaging means, and a third imaging means, and a recognition means for recognizing the products based on the images, wherein the second imaging means and the third imaging means are arranged in different positions from the first imaging means in the vertical and horizontal directions.

[0009] Another information processing device of the present invention includes a first imaging means for imaging products to be settled, a second imaging means and a third imaging means provided at positions different from the first imaging means in the vertical and horizontal directions for imaging the products, and an output means for outputting information about products recognized based on images captured by the first imaging means, the second imaging means, and the third imaging means.

[0010] The control method of the present invention is an information processing method in which a computer acquires images of products to be settled captured by a plurality of imaging means, recognizes the products based on the images, notifies the computer if it cannot uniquely identify the product as a result of the recognition, and determines the image to be used for recognition based on the position of the product to be recognized and the position or orientation of the imaging means.

[0011] Another control method of the present invention involves a computer acquiring images of the product to be settled, captured by a first imaging means, a second imaging means, and a third imaging means, recognizing the product based on the images, wherein the second imaging means and the third imaging means are positioned differently from the first imaging means in the vertical and horizontal directions.

[0012] Another control method of the present invention involves a computer capturing an image of the product to be paid for using a first imaging means, capturing an image of the product using a second imaging means and a third imaging means provided at different vertical and horizontal positions from the first imaging means, and outputting information about the product recognized based on the images captured by the first imaging means, the second imaging means, and the third imaging means. [Effects of the Invention]

[0013] According to the present invention, it is possible to solve the problems that arise when identifying a product using multiple images generated by multiple imaging means. [Brief explanation of the drawing]

[0014] [Figure 1] This is a block diagram illustrating a product registration device according to Embodiment 1. [Figure 2] This diagram illustrates an example of a product handling area. [Figure 3] This is a block diagram illustrating the hardware configuration of a computer used in the implementation of a product registration device. [Figure 4] This is a first perspective view illustrating the arrangement of the two recognition units. [Figure 5] This is a plan view corresponding to the arrangement in Figure 4. [Figure 6] This is a second perspective view illustrating the arrangement of the two recognition units. [Figure 7] This is a plan view corresponding to the arrangement in Figure 6. [Figure 8] This is a third perspective view illustrating the arrangement of the two recognition units. [Figure 9] This is a plan view corresponding to the arrangement in Figure 8. [Figure 10] This is a first perspective view illustrating the arrangement of the four recognition units. [Figure 11] This is a plan view corresponding to the arrangement in Figure 10. [Figure 12] This is a second perspective view illustrating the arrangement of the four recognition units. [Figure 13] This is a plan view corresponding to the arrangement in Figure 12. [Figure 14] It is a diagram illustrating the arrangement of eight recognition units. [Figure 15] It is a plan view corresponding to the arrangement of FIG. 14. [Figure 16] It is a flowchart illustrating the flow of processing executed by the product registration device of Embodiment 1. [Figure 17] It is a perspective view illustrating the arrangement of three recognition units. [Figure 18] It is a plan view corresponding to the arrangement of FIG. 16. [Figure 19] It is a perspective view illustrating the arrangement of eight recognition units. [Figure 20] It is a plan view corresponding to the arrangement of FIG. 18. [Figure 21] It is a block diagram illustrating the product registration device according to Embodiment 3. [Figure 22] It is a diagram specifically illustrating the product registration device of Embodiment 3. [Figure 23] It is a flowchart illustrating the flow of processing executed by the product registration device of Embodiment 3. [Figure 24] It is a flowchart illustrating the flow of processing executed by the product registration device of Embodiment 4. [Figure 25] It is a flowchart illustrating the flow of processing executed by the product registration device of Embodiment 5. [Figure 26] It is a block diagram illustrating the settlement system according to Embodiment 6.

Mode for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and the description will be omitted as appropriate.

[0016] [Embodiment 1] FIG. 1 is a block diagram illustrating the product registration device 2000 according to Embodiment 1. In FIG. 1, each block represents a configuration in terms of functional units, not hardware units.

[0017] The product registration device 2000 has multiple imaging units 2020 and recognition units 2030. The imaging units 2020 capture images of products and generate images. Hereinafter, these images will be referred to as product images.

[0018] The imaging range of each imaging unit 2020 includes part or all of the product passage area. The product passage area is the space through which an operator operating the product registration device 2000 passes products in order to register them for payment. Here, the operator may be a store employee or a customer.

[0019] Figure 2 illustrates an example of a product passage area. Platform 20 is a platform on which product baskets and the like are placed. Product passage area 10 is the area where products pass through. Thus, for example, the product passage area is the space on platform 20. However, product passage area 10 is not limited to the space illustrated in Figure 2, and is only necessary as a space through which an operator passes products in order for the imaging unit 2020 to read product information. Furthermore, stores that operate the product registration device 2000 are not limited to stores that use product baskets.

[0020] The recognition unit 2030 recognizes products using product images generated by each imaging unit 2020. Product recognition means identifying a product and registering it as a product to be paid for.

[0021] For example, the recognition unit 2030 identifies a product by analyzing the image of the product information symbol that appears in the product image. A product information symbol is a symbol that recognizes information about a product. Here, a symbol can be a barcode, a two-dimensional code (such as a QR code (registered trademark)), or a string symbol. Note that the string here also includes a sequence of numbers. A product information symbol is a barcode or the like in which information that recognizes product information (such as a product information ID) is encoded, or a string symbol that represents information that recognizes product information.

[0022] For example, the recognition unit 2030 identifies a product by performing object recognition on the image of the product itself that is visible in the product image.

[0023] <Effects and Actions> The product registration device 2000 of this embodiment has multiple imaging units 2020. Therefore, if a product is imaged by any of the imaging units 2020, the recognition unit 2030 may be able to recognize that product. For example, suppose an operator moves a product in a product passage area, and that product is not placed within the imaging range of a certain imaging unit 2020. In this case, if the product registration device 2000 does not have other imaging units 2020, this product will not be recognized by the recognition unit 2030 and will not be registered as a product for settlement. Therefore, the operator will need to repeat the process of having the imaging unit 2020 read the product information. On the other hand, if the product registration device 2000 has other imaging units 2020, there is a possibility that the product will be placed within the imaging range of the other imaging unit 2020, and therefore the recognition unit 2030 may be able to recognize the product.

[0024] As mentioned above, product recognition using product images is performed by analyzing product information symbols or identifying products through object recognition. In order to analyze product information symbols, the product image must contain the product information symbols. In order to identify products through object recognition, the product image must contain characteristic parts of the product. For example, when identifying the type of canned coffee using object recognition, an image showing only the bottom of the can is insufficient; an image showing the label, etc., is required. According to the product registration device 2000 of this embodiment, if a product information symbol or a characteristic part of the product is captured by any of the imaging units 2020, the recognition unit 2030 can recognize that product.

[0025] Therefore, with the product registration device 2000 of this embodiment, the probability of a product being recognized by the recognition unit 2030 is higher compared to the case where there is only one imaging unit 2020. As a result, the probability of a product being registered as a product for settlement is higher, and the efficiency of the process of having the product registration device 2000 recognize the product is increased.

[0026] The product registration device 2000 of this embodiment will be described in more detail below.

[0027] <Example of hardware configuration for product registration device 2000> The product registration device 2000 may be implemented solely by hardware (e.g., hardwired electronic circuits) or by a combination of hardware and software (e.g., a combination of electronic circuits and a program to control them).

[0028] The product registration device 2000 is implemented using a dedicated terminal, such as a cash register terminal. However, the product registration device 2000 may also be implemented using various general-purpose computers, such as PCs (Personal Computers) or server machines, instead of such dedicated terminals.

[0029] Figure 3 is a block diagram illustrating the hardware configuration of the computer 1000 used in the implementation of the product registration device 2000. The computer 1000 has a bus 1020, a processor 1040, memory 1060, storage 1080, and an input / output interface 1100. The bus 1020 is a data transmission path for the processor 1040, memory 1060, storage 1080, and input / output interface 1100 to send and receive data to and from each other. However, the method of connecting the processor 1040 and the others is not limited to bus connection. The processor 1040 is a processing unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 1060 is a memory such as RAM (Random Access Memory) or ROM (Read Only Memory). The storage 1080 is a storage device such as a hard disk, SSD (Solid State Drive), or memory card. The storage 1080 may also be a memory such as RAM or ROM. Storage 1080 stores various data and programs.

[0030] The input / output interface 1100 is an interface for connecting the computer 1000 with input / output devices. The computer 1000 is connected to a keyboard, mouse, or display, etc., via the input / output interface 1100.

[0031] <<Hardware configuration of imaging unit 2020>> For example, the imaging unit 2020 is implemented by a camera 90 having an image sensor. Alternatively, the imaging unit 2020 may be implemented by a barcode reader. In this case, the imaging unit 2020 generates data representing the barcode pattern by shining light onto the product and receiving the reflected light with a photodetector. The product image also includes this data representing the barcode pattern.

[0032] <Arrangement of imaging unit 2020> The number of imaging units 2020 can be any number, two or more. Below, specific arrangements of imaging units 2020 are provided as examples for the cases of 2, 4, and 8 imaging units. The distance between imaging units 2020 in each arrangement is arbitrary; for example, this distance is 5 cm.

[0033] <<Placement 1>> Figure 4 is a first perspective view illustrating the arrangement of the two imaging units 2020. In Figure 4, the x-direction is the direction in which the operator moves the product. The y-direction is the depth direction of the stand 20. The z-direction is the vertical direction.

[0034] Figure 5 is a plan view corresponding to the arrangement in Figure 4. Figure 5(a) is the yz plan view of Figure 4, and Figure 5(b) is the xy plan view of Figure 4. The imaging range 30 represents the imaging range of the imaging unit 2020. The imaging direction 40 is the direction that starts from the imaging unit 2020 and passes through the center of the imaging range 30 of the imaging unit 2020. Hereafter, the imaging direction 40 of the imaging unit 2020 will also be referred to as the "orientation of the imaging unit 2020".

[0035] As shown in Figure 5(a), the imaging units 2020-1 and 2020-2 are located at different positions in the y-direction. Specifically, the imaging unit 2020-1 is located at one end of the base 20 in the y-direction, and the imaging unit 2020-2 is located at the other end of the base 20 in the y-direction. On the other hand, the imaging units 2020-1 and 2020-2 are located at the same position in the z-direction. Both are mounted on the base 20. Furthermore, as shown in Figure 5(b), the imaging units 2020-1 and 2020-2 are located at the same position in the x-direction.

[0036] As shown in Figure 5(a), the orientation of both the imaging unit 2020-1 and the imaging unit 2020-2 is such that they are looking up at the product passage area 10 from diagonally below in a yz-plane view. Furthermore, as shown in Figure 5(b), it is preferable that a portion of the imaging range 30 of the imaging unit 2020-1 and the imaging unit 2020-2 overlaps with each other.

[0037] This arrangement allows the product to be imaged from multiple different directions in the y-direction. Therefore, compared to the case where the product is imaged from only one direction, the probability of product information symbols or distinctive parts of the product being captured in the product image increases, thus increasing the probability that the recognition unit 2030 can recognize the product.

[0038] Note that the orientation of each imaging unit 2020 in the arrangements shown in Figures 4 and 5 may be in the z-direction.

[0039] <<Placement 2>> Figure 6 is a second perspective view illustrating the arrangement of the two imaging units 2020. Here, the holding unit 2040 is a member for holding the imaging units 2020.

[0040] Figure 7 is a plan view corresponding to the arrangement in Figure 6. Figure 7(a) is the yz plan view of Figure 6, and Figure 7(b) is the xy plan view of Figure 6. As shown in Figure 7(a), the y-direction positions of imaging unit 2020-1 and imaging unit 2020-2 are different, and their positional relationship is the same as in Figure 4. Also, the z-direction positions of imaging unit 2020-1 and imaging unit 2020-2 are the same, but they are different from the positions in Figure 4. Specifically, imaging unit 2020-1 and imaging unit 2020-2 are located above the product passage area 10. As shown in Figure 7(b), the x-direction positions of imaging unit 2020-1 and imaging unit 2020-2 are the same.

[0041] As shown in Figure 7(a), the orientation of both the imaging unit 2020-1 and the imaging unit 2020-2 is such that they are looking down at the product passage area 10 from diagonally above in a yz-plane view. Furthermore, as shown in Figure 7(b), it is preferable that the imaging ranges 30 of the imaging unit 2020-1 and the imaging unit 2020-2 overlap in a portion of the area.

[0042] This arrangement allows the product to be imaged from multiple different directions in the y-direction. Therefore, compared to the case where the product is imaged from only one direction, the probability of product information symbols or distinctive parts of the product being captured in the product image increases, thus increasing the probability that the recognition unit 2030 can recognize the product.

[0043] Note that the orientation of each imaging unit 2020 in the arrangements shown in Figures 6 and 7 may be in the -z direction.

[0044] <<Placement 3>> Figure 8 is a third perspective view illustrating the arrangement of the two imaging units 2020. Figure 9 is a plan view corresponding to the arrangement in Figure 8. Figure 9(a) is a yz plan view of Figure 8, and Figure 9(b) is an xy plan view of Figure 8. As shown in Figure 9(a), the y-direction positions of imaging unit 2020-1 and imaging unit 2020-2 are different, and their positional relationship is the same as in Figure 4. Also, the z-direction positions of imaging unit 2020-1 and imaging unit 2020-2 are different. Specifically, imaging unit 2020-1 is installed on the base 20, and imaging unit 2020-2 is held by the holding unit 2040 and positioned above the product passage area 10. As shown in Figure 9(b), the x-direction positions of imaging unit 2020-1 and imaging unit 2020-2 are different.

[0045] As shown in Figure 9(a), the orientations of imaging units 2020-1 and 2020-2 are opposite each other in a yz-plane view, with the product passage area 10 in between. Also, as shown in Figure 9(b), the orientations of imaging units 2020-1 and 2020-2 are opposite each other in an xy-plane view. Preferably, the imaging ranges 30 of imaging units 2020-1 and 2020-2 overlap in a portion of the area.

[0046] This arrangement allows the product to be imaged from multiple different directions in the y-direction. Therefore, compared to when the product is imaged from only one direction, the probability of product information symbols or distinctive parts of the product appearing in the product image is higher, thus increasing the probability that the recognition unit 2030 can recognize the product. Furthermore, since the product is imaged from both above and below, compared to when the product is imaged from only above or below, the probability of product information symbols or distinctive parts of the product appearing in the product image is higher, thus increasing the probability that the recognition unit 2030 can recognize the product.

[0047] In addition, in the arrangements shown in Figures 8 and 9, the orientation of the imaging unit 2020-1 may be in the -z direction, and the orientation of the imaging unit 2020-2 may be in the z direction.

[0048] <<Placement 4>> Figure 10 is a first perspective view illustrating the arrangement of the four imaging units 2020. Figure 11 is a plan view corresponding to the arrangement in Figure 10. Figure 11(a) is a yz plan view of Figure 10, and Figure 11(b) is an xy plan view of Figure 10. As shown in Figure 11(a), in the y direction, imaging units 2020-1 and 2020-2 are in the same position, and imaging units 2020-3 and 2020-4 are in the same position. Specifically, the y-direction positions of imaging units 2020-1 and 2020-2 are at one end of the base 20, and the y-direction positions of imaging units 2020-3 and 2020-4 are at the other end of the base 20. In the z direction, imaging units 2020-1 and 2020-4 are in the same position, and imaging units 2020-2 and 2020-3 are in the same position. Specifically, imaging units 2020-1 and 2020-4 are held by the holding unit 2040 and positioned above the product passage area 10, while imaging units 2020-2 and 2020-3 are installed on the base 20. As shown in Figure 11(b), the x-direction position of each imaging unit 2020 is the same.

[0049] The imaging directions 40 of imaging units 2020-1 and 2020-3 are opposite to each other. Also, the imaging directions 40 of imaging units 2020-2 and 2020-4 are opposite to each other. Preferably, the imaging directions 40 of imaging units 2020-1 to 2020-4 are directed toward the center point of imaging units 2020-1 to 2020-4, respectively. Furthermore, it is preferable that the imaging ranges 30 of each imaging unit 2020 overlap with each other in some areas.

[0050] Since the four imaging units 2020 each image the product from a different direction, the blind spots of the imaging units 2020 are reduced compared to the case where there are two imaging units 2020. Therefore, the probability of product information symbols or distinctive parts of the product being captured in the product image is higher compared to the case where there are two imaging units 2020, and thus the probability of the recognition unit 2030 being able to recognize the product is higher.

[0051] Note that the orientation of each imaging unit 2020 in the arrangement shown in Figures 8 and 9 is not limited to the orientations shown in Figures 8 and 9. For example, imaging units 2020-1 and 2020-4 may be facing the z direction, while imaging units 2020-2 and 2020-3 may be facing the -z direction.

[0052] <<Placement 5>> Figure 12 is a second perspective view illustrating the arrangement of the four imaging units 2020. Figure 13 is a plan view corresponding to the arrangement in Figure 12. Figure 13(a) is a yz plan view of Figure 12, and Figure 13(b) is an xy plan view of Figure 12. As shown in Figure 13(a), the positional relationships in the y and z directions are the same as in Figure 11(a). On the other hand, as shown in Figure 13(b), the positional relationships in the x direction are different from those in Figure 11(b). Specifically, the x-direction positions of imaging unit 2020-1 and imaging unit 2020-4 are different from those in Figure 11(b).

[0053] The relationship of the imaging direction 40 for each imaging unit 2020 is the same as the relationship shown in Figures 8 and 9. Furthermore, it is preferable that the imaging ranges 30 of each imaging unit 2020 overlap with each other in some areas.

[0054] The arrangements in Figures 12 and 13 differ from those in Figures 10 and 11, as the product is imaged from different directions in the x-direction. Therefore, compared to the arrangements in Figures 10 and 11, the probability of product information symbols or distinctive parts of the product being captured in the product image is higher, thus increasing the probability that the recognition unit 2030 can recognize the product.

[0055] Note that the orientation of each imaging unit 2020 in the arrangement shown in Figures 12 and 13 is not limited to the orientations shown in Figures 12 and 13. For example, imaging unit 2020-1 and imaging unit 2020-2 may be oriented in corresponding directions, while imaging unit 2020-3 and imaging unit 2020-4 may be oriented in opposing directions.

[0056] <<Placement 6>> Figure 14 illustrates the arrangement of the eight imaging units 2020. Figure 15 is a plan view corresponding to the arrangement in Figure 14. Figure 15(a) is the yz plan view of Figure 14, and Figure 15(b) is the xy plan view of Figure 14. As these figures show, the positional relationships from imaging unit 2020-1 to imaging unit 2020-4, and from imaging unit 2020-5 to imaging unit 2020-8 are the same as the positional relationships in the arrangements in Figures 10 and 11. However, as shown in Figure 15(b), the positions in the x-direction differ between imaging unit 2020-1 to imaging unit 2020-4 and imaging unit 2020-5 to imaging unit 2020-8.

[0057] The imaging directions 40 of imaging units 2020-1 and 2020-7, 2020-2 and 2020-8, 2020-3 and 2020-5, and 2020-4 and 2020-6 are opposite each other. Preferably, the imaging direction 40 of each imaging unit 2020 points toward the center position of all imaging units 2020. Furthermore, it is preferable that the imaging ranges 30 of each imaging unit 2020 overlap with each other in some areas.

[0058] As shown in Figures 14 and 15, the blind spots of the imaging unit 2020 are reduced compared to the cases with two or four imaging units 2020. Therefore, the probability of product information symbols or distinctive parts of the product being captured in the product image increases, thus increasing the probability that the recognition unit 2030 can recognize the product.

[0059] <<Regarding the holding part 2040>> The holding portion 2040 is preferably made of a material that allows visible light to pass through. For example, this material may be a rod, column, or plate made of transparent plastic or glass. By allowing visible light to pass through the holding portion 2040, it is possible to prevent the holding portion 2040 from blocking light from reaching the product. As a result, it is possible to prevent a decrease in the accuracy of product recognition by the recognition portion 2030.

[0060] <Processing flow> Figure 16 is a flowchart illustrating the processing flow performed by the product registration device 2000 of Embodiment 1. The imaging unit 2020 generates a product image (S102). The recognition unit 2030 recognizes a product using the product image (S104).

[0061] <Details of the processing performed by the imaging unit 2020> The imaging unit 2020 generates a product image (S102). The imaging unit 2020 may capture a still image or a video. In the latter case, the operator image is each frame that makes up the video.

[0062] The timing of imaging by the imaging unit 2020 varies. For example, the imaging unit 2020 may perform imaging at the time when the operator makes the product registration device 2000 recognize a product, and before and after that time. For example, an infrared sensor for detecting people may be installed near the product registration device 2000. The product registration device 2000 can determine that an operator is near the product registration device 2000 by receiving a notification from this infrared sensor. Therefore, for example, the imaging unit 2020 may perform imaging from the time the infrared sensor detects the presence of an operator near the product registration device 2000 until the infrared sensor no longer detects the operator.

[0063] For example, the imaging unit 2020 may perform repeated imaging periodically. The frequency at which the imaging unit 2020 performs repeated imaging is, for example, 1 / 30th of a second, which is the same as the frame rate of a typical video.

[0064] <Details of the processing performed by the recognition unit 2030> The recognition unit 2030 recognizes products using product images generated by the imaging unit 2020 (S104). For example, the recognition unit 2030 identifies products by detecting product information symbols in each product image generated by each imaging unit 2020 and analyzing the detected product information symbols. Alternatively, the recognition unit 2030 identifies products by performing object recognition on the products themselves that appear in the product images generated by each imaging unit 2020. If a product can be identified from any of the product images, the identified product is registered as a product to be settled.

[0065] Here, specific results obtained for each of multiple product images may indicate different products. For example, suppose a specific result using a product image generated by one imaging unit 2020 indicates that "the product is X," while a specific result using a product image generated by another imaging unit 2020 indicates that "the product is Y." In this case, the recognition unit 2030 will identify the product using the most likely result. For example, the recognition unit 2030 will adopt the result with the most product images that produce the same identification result. For example, suppose the analysis of three product images each indicates that "the product is X," while the analysis of five other product images each indicates that "the product is Y." In this case, the recognition unit 2030 will adopt the result with the most product images, "the product is Y," and register product Y as the product to be settled. However, the recognition unit 2030 may issue a warning if it is unable to uniquely identify the product in this way.

[0066] [Embodiment 2] The product registration device 2000 of Embodiment 2 is shown in Figure 1, similar to the product registration device 2000 of Embodiment 1. The product registration device 2000 of Embodiment 2 is the same as the product registration device 2000 of Embodiment 1, except that the arrangement of the imaging unit 2020 is different.

[0067] In the second embodiment, the product registration device 2000 has at least two of its multiple imaging units 2020 arranged to image products that are facing in different directions. The arrangement of the imaging units 2020 will be described in detail below.

[0068] <Placement 1> Figure 17 is a perspective view illustrating the arrangement of three imaging units 2020. The x, y, and z directions are the directions in which the operator moves the product, the depth direction of the stand 20, and the vertical direction, respectively. The imaging range 30 is the imaging range of the imaging unit 2020. The imaging direction 40 is the direction starting from the imaging unit 2020 and passing through the center of the imaging range 30 of that imaging unit 2020.

[0069] As shown in Figure 17, the imaging directions 40 of each imaging unit 2020 are oriented differently from each other. Furthermore, each imaging unit 2020 is housed in the same housing 100.

[0070] Figure 18 is a plan view corresponding to the arrangement in Figure 17. Figure 18(a) is the yz plan view of Figure 17, and Figure 18(b) is the xy plan view of Figure 17. As shown in Figure 18(b), each imaging unit 2020 images the products 50 which are in different directions from each other.

[0071] By arranging the imaging units 2020 in this manner, the product is imaged from multiple different directions, increasing the probability that product information symbols or distinctive parts of the product will be captured in the product image, and thus increasing the probability that the recognition unit 2030 can recognize the product. Furthermore, as shown in the arrangements in Figures 17 and 18, multiple imaging units 2020 can be placed close together, allowing them to be housed in the same housing. This simplifies the installation of the imaging units 2020.

[0072] <Placement 2> Figure 19 is a perspective view illustrating the arrangement of eight imaging units 2020. In Figure 19, imaging units 2020-1 to 2020-3 are housed in the same housing 100-1. Similarly, imaging units 2020-4 to 2020-6 are housed in the same housing 100-2. Housings 100-1 and 100-2 are installed facing each other. Imaging unit 2020-7 is installed on a stand 20. On the other hand, imaging unit 2020-8 is installed above the product passage area 10. Although not shown in the figure, imaging unit 2020-8 is held by a holding unit 2040.

[0073] Figure 20 is a plan view corresponding to the arrangement in Figure 19. Figure 20(a) is the yz plan view of Figure 19, and Figure 20(b) is the xy plan view of Figure 19. As shown in Figure 20(b), imaging units 2020-1 to 2020-3 image products 50 that are in different directions from each other, similar to the arrangement in Figure 17. Imaging units 2020-4 to 2020-6 also image products 50 that are in different directions from each other. Imaging units 2020-2 and 2020-5, and imaging units 2020-3 and 2020-6 each image product 50 at the same location from different directions from each other. Imaging units 2020-1, 2020-4, 2020-7, and 2020-8 also image product 50 at the same location from different directions from each other.

[0074] According to the arrangement in Figures 19 and 20, the product 50 can be imaged from various directions by more imaging units 2020 than in Figures 17 and 18. This increases the probability that product information symbols or distinctive parts of the product will be captured in the product image, and thus increases the probability that the recognition unit 2030 can recognize the product.

[0075] The hardware configuration of the product registration device 2000 in Embodiment 2 is the same as that of the product registration device 2000 in Embodiment 1, except for the arrangement of the imaging unit 2020.

[0076] Furthermore, the processing performed by each functional component of the product registration device 2000 in Embodiment 2 is the same as the processing performed by each functional component of the product registration device 2000 in Embodiment 1.

[0077] <Effects and Actions> According to the product registration device 2000 of this embodiment, since multiple imaging units 2020 image products from different directions, the probability of product information symbols or characteristic parts of the product being captured in the product image is higher compared to the case where there is only one imaging unit 2020. Therefore, the probability of the recognition unit 2030 being able to recognize the product is higher. Furthermore, in the product registration device 2000 of this embodiment, multiple product registration devices 2000 are installed in the same or nearby locations, and these imaging units 2020 image products from different directions. Therefore, these imaging units 2020 can be housed in a common housing, making the installation of the imaging units 2020 easier.

[0078] [Embodiment 3] Figure 21 is a block diagram illustrating a product registration device 2000 according to Embodiment 3. In Figure 21, each block represents a functional unit configuration, not a hardware unit configuration.

[0079] The product registration device 2000 of Embodiment 3 has a control unit 2060. The control unit 2060 captures the operator's movements and generates an image. Hereinafter, the image generated by the control unit 2060 will be referred to as the operator image. The control unit 2060 then uses the operator image to control the imaging unit 2020.

[0080] <Hardware configuration of product registration device 2000> The hardware configuration of the product registration device 2000 in Embodiment 3 is shown in Figure 3, similar to the hardware configuration of the product registration device 2000 in Embodiment 1.

[0081] The control unit 2060 has an image sensor for generating an image of the operator. For example, the control unit 2060 is implemented using a camera with an image sensor. Figure 22 is a diagram specifically illustrating the product registration device 2000 of Embodiment 3. In the product registration device 2000 of Figure 22, the camera 110 for realizing the control unit 2060 is installed on the holding unit 2040. However, the installation position of the camera 110 is not limited to the position shown in Figure 22.

[0082] The control unit 2060 may have one camera or multiple cameras.

[0083] Here, the resolution of the image sensor in the control unit 2060 may be lower than the resolution of the image sensor in the imaging unit 2020. This is because, as will be described later, the required resolution for the operator image, which only needs to be used for detecting the position of products and product information symbols, is lower than the required resolution for the product image, which needs to be used for identifying products. By lowering the resolution of the image sensor in the control unit 2060 in this way, the manufacturing cost of the product registration device 2000 can be reduced. However, the resolution of the image sensor in the control unit 2060 may be higher than or equal to the resolution of the image sensor in the imaging unit 2020.

[0084] The storage 1080 of Embodiment 3 further stores program modules for realizing each functional component of Embodiment 3. The processor 1040 then executes these program modules to realize the functions of each functional component of Embodiment 4.

[0085] <Processing flow> Figure 23 is a flowchart illustrating the processing flow performed by the product registration device 2000 of Embodiment 3. The control unit 2060 generates an operator image (S202). The control unit 2060 controls the imaging unit 2020 based on the operator image (S204). The imaging unit 2020 images the product and generates a product image based on the control by the control unit 2060 (S206). The recognition unit 2030 recognizes the product using the generated product image (S208).

[0086] <Details of the processing performed by the control unit 2060> The control unit 2060 generates operator images (S202). The control unit 2060 may capture still images or capture video. In the latter case, the operator images are each frame that makes up the video.

[0087] The timing at which the control unit 2060 performs imaging varies. This timing is the same as the timing at which the imaging unit 2020 performs imaging, as described in Embodiment 1.

[0088] Furthermore, the control unit 2060 controls the imaging unit 2020 using the operator's image (S204). There are various specific methods by which the control unit 2060 controls the imaging unit 2020. These methods are explained below as examples.

[0089] <<Control Method 1>> The control unit 2060 controls the timing at which each imaging unit 2020 captures images of products. First, the control unit 2060 uses the operator image to detect the product and calculate the direction and speed of its movement. Here, techniques such as object recognition can be used to detect the product from the image. The control unit 2060 also calculates the direction and speed of the product's movement based on the change in the position of the product in multiple operator images.

[0090] The control unit 2060 then uses the time of image capture of the operator, the direction of movement of the product, and the speed of movement of the product to calculate the timing at which the product is positioned within the imaging range of each imaging unit 2020. The control unit 2060 then causes each imaging unit 2020 to image the product at the timing calculated for each imaging unit 2020. In this way, each imaging unit 2020 can be operated at a timing when the recognition unit 2030 has a high probability of recognizing the product, thus increasing the probability that the recognition unit 2030 can recognize the product.

[0091] Furthermore, the control unit 2060 does not need to perform imaging of the product in the imaging unit 2020 if the product is not within the imaging range. This reduces the power consumption of the imaging unit 2020 and prevents deterioration of the imaging unit 2020.

[0092] In this case, when the product is imaged by the imaging unit 2020, the product is held in the operator's hand, and therefore a portion of the product's surface is covered by the operator's hand. As a result, the operator's image may sometimes show the product with a portion missing.

[0093] In such cases, one possible approach is to interpolate the parts not visible in the operator's image to recreate the entire product, and then control the imaging timing of the imaging unit 2020. However, it is preferable for the control unit 2060 to determine the imaging timing of the imaging unit 2020 using the area of ​​the product visible in the operator's image, without performing such interpolation. This is because there is a high probability that parts covered by the operator's hand will not be visible in the product image generated by the imaging unit 2020, and therefore it is not appropriate to have the imaging unit 2020 image the product at a time when such parts are present.

[0094] For example, the control unit 2060 determines the imaging timing for each imaging unit 2020 by considering only the portion of the product's area that is visible in the operator's image as representing that product. In this way, the control unit 2060 can cause the imaging unit 2020 to take an image when the portion of the product visible in the product image (the part not covered by hands, etc.) is within the imaging range. Furthermore, the control unit 2060 can cause only the imaging unit 2020 to take an image when the portion of the product visible in the product image is within the imaging range.

[0095] Furthermore, the product movement speed may be set as a fixed value in advance, rather than being calculated using the operator's image. For example, before starting operation of the product registration device 2000, the developers of the product registration device 2000 conduct operational tests to enable the product registration device 2000 to recognize products. In this way, the developers repeatedly measure the product movement speed when enabling the product registration device 2000 to recognize the products, and use the results to determine the fixed value mentioned above. In this case, the control unit 2060 performs the same control as described above using the operator image acquisition time, the product movement direction determined from the operator image, and the product movement speed set as a fixed value in advance.

[0096] Furthermore, the direction of movement of the products may also be set as a fixed value in advance. For example, this fixed value is the direction of the longer side of the platform 60.

[0097] Each of the fixed values ​​mentioned above may be pre-set in the control unit 2060, or it may be stored in a memory unit accessible from the control unit 2060. In the latter case, the control unit 2060 reads each fixed value from this memory unit and uses it.

[0098] Furthermore, if the position and imaging direction of the camera etc. of the control unit 2060, and the position and recognition direction of the imaging unit 2020 are fixed, it is possible to pre-determine which imaging unit 2020 should perform imaging when a product is detected at a specific location in the operator image, in relation to the location of the product. Therefore, this pre-determined control method is stored in a memory unit accessible from the control unit 2060. In this case, when the control unit 2060 detects a product from the operator image, it controls the imaging unit 2020 by referring to the information stored in this memory unit.

[0099] <<Control Method 2>> The control unit 2060 calculates the position, speed, and direction of movement of the product information symbol and controls the imaging unit 2020 based on these. For example, the control unit 2060 causes the imaging unit 2020 to take an image when the product information symbol is positioned within the imaging range of the imaging unit 2020. Alternatively, the control unit 2060 may control the imaging unit 2020 so that only the imaging unit 2020 in which the product information symbol is positioned within the imaging range takes an image, and the imaging unit 2020 in which the product information symbol is not positioned within the imaging range does not take an image. Here, the method for calculating the position, speed, and direction of movement of the product information symbol is the same as the method for calculating the position, speed, and direction of movement of the product. Furthermore, the method for determining the timing for each imaging unit 2020 to perform imaging, or for having only some of the imaging units 2020 perform imaging, based on the position, speed, and direction of movement of the product information symbol, is the same as the method for determining the timing for each imaging unit 2020 to perform imaging, or for having only some of the imaging units 2020 perform imaging, based on the position, speed, and direction of movement of the product.

[0100] In this case, the product information symbol may not be visible in the operator image. For example, if the control unit 2060 images the operator's movements from an oblique angle above, the product information symbol attached to the bottom of the product may not be visible. Therefore, if the control unit 2060 detects a product from the operator image but cannot detect the product information symbol, it may control the imaging units 2020 to take an image, while preventing other imaging units 2020 from taking an image. The imaging units 2020 located in the blind spots of the control unit 2060's camera can be determined in advance from the position and imaging direction of the control unit 2060, as well as the arrangement of each imaging unit 2020.

[0101] This method of controlling the imaging unit 2020 based on the position of the product information symbol allows for more appropriate timing of imaging by the imaging unit 2020 compared to a method of controlling the imaging unit 2020 based on the position of the product, and also allows imaging to be performed only by the imaging unit 2020 that can read the product information symbol. In this case, the recognition unit 2030 recognizes the product by analyzing the product information symbol.

[0102] <Effects and Actions> According to this embodiment, the imaging unit 2020 is controlled based on the operator's image. As a result, as described above, the probability of the recognition unit 2030 being able to recognize the product can be increased, the power consumption of the imaging unit 2020 can be reduced, and deterioration of the imaging unit 2020 can be prevented.

[0103] [Embodiment 4] The product registration device 2000 of Embodiment 4 is represented by Figure 21, similar to the product registration device 2000 of Embodiment 3. Except for the points described below, the product registration device 2000 of Embodiment 4 is the same as the product registration device 2000 of Embodiment 3.

[0104] In Embodiment 4, the control unit 2060 determines one or more product images to be used for product recognition based on the operator image and the position and orientation of each imaging unit 2020. The recognition unit 2030 recognizes the product using the product images determined by the control unit 2060.

[0105] <Hardware Configuration> The hardware configuration of the product registration device 2000 in Embodiment 4 is shown in Figure 3, similar to the hardware configuration of the product registration device 2000 in Embodiment 1. The storage 1080 in Embodiment 4 further stores program modules for realizing each functional component of Embodiment 4. The processor 1040 then executes these program modules to realize the functions of each functional component of Embodiment 4.

[0106] <Processing flow> Figure 24 is a flowchart illustrating the processing flow performed by the product registration device 2000 of Embodiment 4. Each imaging unit 2020 generates a product image (S302). The control unit 2060 generates an operator image (S304). The control unit 2060 uses the operator image to determine the product image to be used for product recognition (S306). The recognition unit 2030 recognizes the product using the determined product image (S308).

[0107] <Details of the processing performed by the control unit 2060> The control unit 2060 determines the product image using the operator image (S306). To do this, the control unit 2060 calculates the timing at which the product is placed within the imaging range of each imaging unit 2020, in the same manner as the control unit 2060 in Embodiment 3. However, in this embodiment, the method of using the calculated timing differs from that in Embodiment 3. This will be explained in detail below.

[0108] <<Control Method 1>> For example, the control unit 2060 determines the timing at which a product is placed within the imaging range for each imaging unit 2020, using a method similar to the control method 1 described in Embodiment 3. Then, for each imaging unit 2020, the control unit 2060 identifies the product image captured at the timing when the product was placed within the imaging range from among the product images generated by that imaging unit 2020. The control unit 2060 then decides to use each identified product image for product recognition.

[0109] For example, the control unit 2060 identifies the imaging unit 2020 in which the product is placed within the imaging range, using the same method as control method 1 described in Embodiment 3. The control unit 2060 then decides to use the product image generated by the imaging unit 2020 in which the product is placed within the imaging range for product recognition.

[0110] <<Control Method 2>> For example, the control unit 2060 determines the timing at which the product information symbol is placed within the imaging range using a method similar to the control method 2 described in Embodiment 3. Then, for each imaging unit 2020, the control unit 2060 identifies the product image captured at the timing when the product information symbol is placed within the imaging range, from among the product images generated by that imaging unit 2020. The control unit 2060 then decides to use each identified product image for product recognition.

[0111] For example, the control unit 2060 identifies the imaging unit 2020 in which the product information symbol is placed within the imaging range, using the same method as control method 2 described in Embodiment 3. The control unit 2060 then decides to use the product image generated by the imaging unit 2020 in which the product information symbol is placed within the imaging range for product recognition.

[0112] <Effects and Actions> According to this embodiment, for the same reasons as in Embodiment 3, the probability of the product registration device 2000 being able to recognize a product is increased. Furthermore, according to this embodiment, the control unit 2060 does not need to control the imaging unit 2020. Therefore, the control of the imaging unit 2020 by the product registration device 2000 is simplified.

[0113] [Embodiment 5] The product registration device 2000 of Embodiment 5 is represented in Figure 21, similar to the product registration device 2000 of Embodiment 4. Except for the points described below, the product registration device 2000 of Embodiment 5 is the same as the product registration device 2000 of Embodiment 4.

[0114] The control unit 2060 in Embodiment 4 calculates the timing at which a product or product information symbol is placed within the imaging range of each imaging unit 2020. However, since the timing calculated by the control unit 2060 is a predicted value, the actual timing at which a product or product information symbol is placed within the imaging range of each imaging unit 2020 may differ from this predicted value.

[0115] Furthermore, the control unit 2060 of Embodiment 4 predicts which imaging unit 2020 will place a product or product information symbol within its imaging range by calculating the product's direction of movement, etc. However, since the product's direction of movement can change, the product may not be placed within the imaging range of the imaging unit 2020 that was predicted to place the product within its imaging range, or the product may be placed within the imaging range of the imaging unit 2020 that was predicted not to place the product within its imaging range.

[0116] Therefore, the control unit 2060 of Embodiment 5 assigns weights to the product images captured by the imaging unit 2020. Then, the recognition unit 2030 uses each weighted product image to recognize the product.

[0117] <Processing flow> Figure 25 is a flowchart illustrating the processing flow performed by the product registration device 2000 of Embodiment 5. The processing performed in S302 and S304 is the same as in Figure 23. The control unit 2060 assigns weights to each product image (S402). The recognition unit 2030 recognizes the product using the weighted product images (S404).

[0118] <Details of the processing performed by the control unit 2060> In Embodiment 5, the control unit 2060 assigns weights to each product image using the operator image (S402). Specifically, the control methods described in Embodiments 3 and 4 are used to assign weights to the product images. For example, the control unit 2060 makes the weight of the imaging unit 2020 in which it is predicted that "a product or product information symbol is placed within the imaging range" greater than the weight of the imaging unit 2020 in which it is predicted that "a product or product information symbol is not placed within the imaging range". Furthermore, the control unit 2060 assigns a greater weight to product images that are captured closer to the time when it is predicted that the product or product information symbol will be placed within the imaging range.

[0119] Here, the control unit 2060 assigns a weight to the product image i based on whether or not the product or product information symbol is placed within the imaging range, and the control unit 2060 assigns a weight to the product image i based on the discrepancy between the timing at which the product or product information symbol is placed within the imaging range and the time at which the product image is captured. In this case, for example, the control unit 2060 assigns a weight of ai*bi to the product image i. However, the control unit 2060 may assign only ai or bi to the product image i.

[0120] <Details of the processing performed by the recognition unit 2030> The recognition unit 2030 recognizes products using weighted product images (S404). Here, the recognition unit 2030 uses the weights assigned to product images as an indicator of the likelihood of the product being recognized using that product image. Specifically, for each product image that shows the same product, the recognition unit 2030 identifies the product shown in that product image. If the recognition unit 2030 is unable to uniquely identify the product, it uses the weights assigned to each product image as the likelihood of identification based on that product image to uniquely identify the product.

[0121] For example, suppose that among the multiple product images generated for a single product, some product images are identified as "product X" through object recognition, etc. On the other hand, suppose that other product images are identified as "product Y" through the results of object recognition, etc. In this case, the recognition unit 2030 calculates the likelihood of these two identifications using the weights assigned to each product image, and uses the result with the higher likelihood as the product identification result. For example, the recognition unit 2030 uses the weight of the product image identified as "product X" and the weight of the product image identified as "product Y" as the likelihood of their respective identifications. The recognition unit 2030 then uses the identification with the larger weight as the final identification result.

[0122] In the example above, let's assume there are multiple product images that have been identified as "product X" and multiple product images that have been identified as "product Y". In this case, the recognition unit 2030 compares the sum of the weights of the product images identified as "product X" with the sum of the weights of the product images identified as "product Y", and the identification with the larger sum of weights is the final identification result.

[0123] <Effects and Actions> According to the product registration device 2000 of this embodiment, by weighting each product image using a predicted timing value for when the product is placed within the imaging range of the imaging unit 2020, products can be uniquely identified with high accuracy. Therefore, the accuracy of product recognition by the product registration device 2000 is improved.

[0124] [Embodiment 6] Figure 26 is a block diagram illustrating the settlement system 4000 according to Embodiment 6. In Figure 1, each block represents a functional unit configuration, not a hardware unit configuration.

[0125] The settlement system 4000 includes one of the product registration devices 2000 from Embodiment 1 to Embodiment 5, and a settlement device 3000. The settlement device 3000 is a device that performs settlement processing for products recognized by the recognition unit 2030.

[0126] The product registration device 2000 generates settlement information for products that have been registered as settlement targets by recognition by the recognition unit 2030. Here, a single settlement process may include multiple products as settlement targets. For example, after receiving an operation to start the settlement target registration process, the product registration device 2000 registers one or more products recognized by the recognition unit 2030 as settlement targets in a single settlement process, between receiving an operation to start the settlement target registration process and receiving an operation to end the settlement target registration process. The settlement information for a particular settlement process indicates the ID of each product registered as a target of that settlement process. The settlement information may also further indicate the transaction number, the price of each product, and the total amount.

[0127] The settlement device 3000 is used for processing the settlement of goods registered as items to be settled. Specifically, the settlement device 3000 performs processes such as displaying the total amount, accepting payment, counting the amount of money paid, returning change, and issuing a receipt.

[0128] The settlement device 3000 may be provided as an integral part of the product registration device 2000, or it may be provided as a separate unit. If the settlement device 3000 is provided as an integral part of the product registration device 2000, the computer 1000 has the function of operating not only as the product registration device 2000, but also as the settlement device 3000.

[0129] If the settlement device 3000 is provided separately from the product registration device 2000, the settlement device 3000 may be implemented as a dedicated terminal such as a cash register terminal, or as a general-purpose computer. The hardware configuration of the computer that implements the settlement device 3000 is similar to, for example, the hardware configuration of computer 1000 in Figure 3. Furthermore, the input / output interface of the computer that implements the settlement device 3000 is connected to a device for customers to insert money, a device for refunding change, and a device for issuing receipts.

[0130] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0131] Examples of reference formats are provided below. 1. Multiple imaging means for capturing images of products and generating product images are located at different positions from each other. A product registration device having recognition means for recognizing a product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, which is captured in the product image generated by each of the aforementioned imaging means. 2. The product registration device according to 1, wherein the first imaging means and the second imaging means image the product from different directions. 3. The product registration device according to 1. or 2., wherein the first imaging means and the second imaging means are positioned opposite each other across a product passage area through which the product passes. 4. Multiple imaging means for capturing images of products and generating product images, Each of the imaging means includes a recognition means that recognizes a product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, which is captured in the product image generated by the imaging means. Each of the aforementioned imaging means is a product registration device that images products located in different directions. 5. The product registration device according to any one of 1 to 4, wherein each imaging means includes in its imaging range part or all of the product passage area through which the product passes. 6. Having a holding means that holds the imaging means and allows visible light to pass through, The visible light that has passed through the holding means passes through the product passage area through which the product passes, as described in any one of 1 to 5 of the product registration device. 7. A product registration device according to any one of 1 to 6, comprising control means for capturing images of the operator's movements to generate an operator image, and using the operator image to cause some of the multiple imaging means to capture images of a product. 8. A control means that captures the operator's movements to generate an operator image, and uses that operator image to determine which product image to use for product recognition from among the product images captured by each imaging means. The product registration device according to any one of 1 to 6, wherein the recognition means recognizes a product using a product image determined by the control means. 9. The system has control means that captures the operator's movements to generate an operator image, and uses that operator image to determine a product image to be used for product recognition. The recognition means is a product registration device according to any one of 1 to 6, which recognizes products using each of the weighted product images. 10. A program that causes a computer to operate as a product registration device as described in any one of items 1 through 9. 11. A control method to be executed by a computer, The imaging means, which are positioned at different locations from each other, perform an imaging step to capture images of the product and generate a product image, A control method comprising: a recognition step of recognizing a product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, which is captured in the product image generated by each of the imaging means. 12. The control method according to 11, wherein the first imaging means and the second imaging means image the product from different directions. 13. The control method according to 11. or 12., wherein the first imaging means and the second imaging means are positioned opposite each other across a product passage area through which the product passes. 14. A control method performed by a computer, comprising an imaging step in which multiple imaging means image a product and generate a product image, The process includes a recognition step of recognizing a product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, which is captured in the product image generated by each of the imaging means, Each of the aforementioned imaging means is a control method for imaging products located in different directions. 15. The control method according to any one of claims 11 to 14, wherein each imaging means includes in its imaging range part or all of the product passage area through which the product passes. 16. Having a holding means that holds the imaging means and allows visible light to pass through, The control method according to any one of 11 to 15, wherein the visible light that has passed through the holding means passes through the product passage area through which the product passes. 17. A control method according to any one of 11 to 16, comprising a control step of capturing the operator's movements to generate an operator image, and using the operator image to cause some of the multiple imaging means to capture an image of a product. 18. A control step that involves capturing the operator's movements to generate an operator image, and using that operator image to determine which product image to use for product recognition from among the product images captured by each imaging means, The control method according to any one of 11 to 16, wherein the recognition step is performed using the product image determined by the control step. 19. The system includes a control step of capturing the operator's movements to generate an operator image, and using that operator image to determine a product image to be used for product recognition. The control method according to any one of 11 to 16, wherein the recognition step is to recognize the product using each of the weighted product images. 20. A settlement system comprising a product registration device described in any one of paragraphs 1 to 9, and a settlement device that performs settlement processing for products recognized by the recognition means of the product registration device. [Explanation of symbols]

[0132] 10 Product Passage Area 20 units 30 Imaging range 40 Imaging direction 50 products 60 units 90 Camera 100 cabinets 110 Camera 1000 calculator 1020 Bus 1040 processor 1060 memory 1080 storage 1100 Input / Output Interface 2000 Product Registration Device 2020 Imaging Department 2030 Recognition part 2040 Holding part 2060 Control Unit

Claims

1. An acquisition means for acquiring images of products to be paid for, captured by multiple imaging means, A recognition means for recognizing the product based on the aforementioned image, The system includes a notification means that provides notification when the recognition result of the recognition means cannot be uniquely identified, The recognition means is an information processing device that determines an image to be used for recognition based on the position of the product to be recognized and the position or orientation of the imaging means.

2. The system further includes a control means that, based on the change in the position of the product in a series of operator images taken of the operator, identifies the timing at which the product in the operator image is positioned within the imaging range of each of the multiple imaging means, and causes each of the multiple imaging means to take an image at the identified timing. The information processing apparatus according to claim 1.

3. The aforementioned plurality of imaging means are positioned the same in one direction in the vertical direction, the direction in which the operator moves the product, and the depth direction, while their positions differ in the other two directions. The information processing apparatus according to claim 1 or 2.

4. The aforementioned plurality of imaging means are positioned the same in two directions: the vertical direction, the direction in which the operator moves the product, and the depth direction, while being positioned differently in the third direction. The information processing apparatus according to any one of claims 1 to 3.

5. The aforementioned recognition means is Based on the position of the product to be recognized and the position or orientation of the imaging means, the timing at which the product enters the imaging range is determined for each of the multiple imaging means. The information processing apparatus according to any one of claims 1 to 4, wherein the image captured by the imaging means at the specified timing is determined to be an image to be used for recognition.

6. Computers Images of the products to be paid for are acquired using multiple imaging devices. Based on the aforementioned image, the product is recognized, If the recognition result cannot be uniquely identified, a notification will be sent. An information processing method for determining an image to be used for recognition based on the position of the product to be recognized and the position or orientation of the imaging means.

7. The aforementioned computer, Based on the changes in the position of the product in multiple time-series images of the operator, the timing at which the product in the operator image is positioned within the imaging range of each of the multiple imaging means is identified, and each of the multiple imaging means is instructed to take an image at the identified timing. The information processing method according to claim 6.

8. The aforementioned plurality of imaging means are positioned the same in one direction in the vertical direction, the direction in which the operator moves the product, and the depth direction, while their positions differ in the other two directions. The information processing method according to claim 6 or 7.

9. The aforementioned plurality of imaging means are positioned the same in two directions: the vertical direction, the direction in which the operator moves the product, and the depth direction, while being positioned differently in the third direction. The information processing method according to any one of claims 6 to 8.

10. The aforementioned computer, Based on the position of the product to be recognized and the position or orientation of the imaging means, the timing at which the product enters the imaging range is determined for each of the multiple imaging means. The information processing method according to any one of claims 6 to 9, wherein the image captured by the imaging means at the specified timing is determined to be the image to be used for recognition.

Images