Information processor and control method

Abstract

To solve the problems that arise when identifying articles using multiple images generated by multiple imaging means.SOLUTION: An information processor includes recognition means for recognizing an object using multiple object images captured by imaging means, and determination means for determining which recognition result to output on the basis of the number of times the same recognition result was recognized when at least one of the recognition results for the object based on each of the multiple object images differs from at least another recognition result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus and a control method. [Background technology]

[0002] Cash registers operated by operators themselves are beginning to be used in stores such as supermarkets. The operator registers the products to be purchased by having the cash register terminal recognize the barcodes attached to the products. The operator then purchases the registered products by inserting the amount displayed on the screen into the cash register terminal.

[0003] An example of a document disclosing technology related to such cash register terminals is Patent Document 1. Patent Document 1 discloses technology that has one scanner and one camera, and uses the results of scanning a product and the results of capturing an image of the product by the camera to determine whether the scanned product is the same as the captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-160493 Summary of the Invention [Problem to be solved by the invention]

[0005] In order to improve the accuracy of product recognition, it is conceivable to identify products using multiple images generated by multiple imaging means.

[0006] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to solve the problems that arise when identifying a product using multiple images generated by multiple imaging means. [Means for solving the problem]

[0007] The first information processing device of the present invention comprises an acquisition means for acquiring images of the product to be checked out that have been captured by a plurality of imaging means, a recognition means for recognizing the product based on the images, and a notification means for issuing a notification when the recognition results based on the images captured by each imaging means differ.

[0008] The second information processing device of the present invention comprises an acquisition means for acquiring images of the product to be checked out that have been captured by a first imaging means, a second imaging means, and a third imaging means, and a recognition means for recognizing the product based on the images, and the second imaging means and the third imaging means are positioned at different positions vertically and horizontally from the first imaging means.

[0009] The third information processing device of the present invention comprises a first imaging means for capturing an image of a commodity to be checked out, a second imaging means and a third imaging means arranged at different vertical and horizontal positions from the first imaging means for capturing an image of the commodity, and an output means for outputting information relating to the commodity recognized based on images captured by the first imaging means, the second imaging means, and the third imaging means.

[0010] In the first control method of the present invention, a computer acquires images of the product to be checked out that have been captured by multiple imaging means, recognizes the product based on the images, and notifies the user if the recognition results based on the images captured by each of the imaging means differ.

[0011] A second control method of the present invention involves a computer acquiring images of the product to be checked out taken by a first imaging means, a second imaging means, and a third imaging means, recognizing the product based on the images, and the second imaging means and the third imaging means being positioned at different vertical and horizontal positions from the first imaging means.

[0012] In a third control method of the present invention, a computer uses a first imaging means to capture an image of a product to be checked out, then uses a second imaging means and a third imaging means that are located at different vertical and horizontal positions from the first imaging means to capture an image of the product, and outputs information about the product recognized based on the images captured by the first imaging means, the second imaging means, and the third imaging means. [Effects of the Invention]

[0013] According to the present invention, it is possible to solve the problem that occurs when identifying a product using a plurality of images generated by a plurality of imaging means. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram illustrating a product registration device according to a first embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of a product passage area. [Figure 3] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer used to implement the product registration device. [Figure 4] FIG. 10 is a first perspective view illustrating the arrangement of two recognition units. [Figure 5] FIG. 5 is a plan view corresponding to the arrangement of FIG. [Figure 6] FIG. 10 is a second perspective view illustrating the arrangement of two recognition units. [Figure 7] FIG. 7 is a plan view corresponding to the arrangement of FIG. 6. [Figure 8] FIG. 10 is a third perspective view illustrating the arrangement of two recognition units. [Figure 9] FIG. 9 is a plan view corresponding to the arrangement of FIG. 8. [Figure 10] FIG. 10 is a first perspective view illustrating an arrangement of four recognition units. [Figure 11] FIG. 11 is a plan view corresponding to the arrangement of FIG. [Figure 12] FIG. 10 is a second perspective view illustrating the arrangement of four recognition units. [Figure 13] FIG. 13 is a plan view corresponding to the arrangement of FIG. 12. [Figure 14] FIG. 10 is a diagram illustrating an example of the arrangement of eight recognition units. [Figure 15] FIG. 15 is a plan view corresponding to the arrangement of FIG. [Figure 16] 1 is a flowchart illustrating a flow of processing executed by the product registration device of the first embodiment. [Figure 17] FIG. 10 is a perspective view illustrating an example of the arrangement of three recognition units. [Figure 18] FIG. 17 is a plan view corresponding to the arrangement of FIG. 16. [Figure 19] FIG. 10 is a perspective view illustrating an example of the arrangement of eight recognition units. [Figure 20] FIG. 19 is a plan view corresponding to the arrangement of FIG. 18. [Figure 21] FIG. 10 is a block diagram illustrating a product registration device according to a third embodiment. [Figure 22] FIG. 10 is a diagram specifically illustrating a product registration device according to a third embodiment. [Figure 23] 11 is a flowchart illustrating a flow of processing executed by a merchandise registration device of a third embodiment. [Figure 24] 10 is a flowchart illustrating a flow of processing executed by a merchandise registration device according to a fourth embodiment. [Figure 25] 13 is a flowchart illustrating a flow of processing executed by the merchandise registration device of the fifth embodiment. [Figure 26] FIG. 10 is a block diagram illustrating an example of a settlement system according to a sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0016] [Embodiment 1] Fig. 1 is a block diagram illustrating a product registration device 2000 according to embodiment 1. In Fig. 1, each block represents a functional configuration rather than a hardware configuration.

[0017] The merchandise registration device 2000 has a plurality of imaging units 2020 and a recognition unit 2030. The imaging units 2020 capture images of merchandise to generate images. Hereinafter, these images will be referred to as merchandise images.

[0018] The imaging range of each imaging unit 2020 includes part or all of the merchandise passage area. The merchandise passage area is a space through which an operator of the merchandise registration device 2000 passes merchandise to be checked out in order to register the merchandise. Here, the operator may be a store clerk or a customer.

[0019] FIG. 2 is a diagram illustrating a product passage area. Platform 20 is a platform on which shopping baskets and the like are placed. Product passage area 10 is a product passage area. In this way, for example, the product passage area is the space above platform 20. However, product passage area 10 is not limited to the space illustrated in FIG. 2 as long as it is a space through which an operator passes products in order to have the image capturing unit 2020 read the product information. Furthermore, stores that operate product registration device 2000 are not limited to stores that use shopping baskets.

[0020] The recognition unit 2030 recognizes the product using the product images generated by each imaging unit 2020. Recognizing the product means identifying the product and registering the product as a payment target.

[0021] For example, the recognition unit 2030 identifies a product by analyzing an image of a product information symbol shown in a product image. The product information symbol is a symbol that recognizes information about the product. Here, the symbol may be a barcode, a two-dimensional code (such as a QR code (registered trademark)), or a character string symbol. Note that the character string referred to here also includes a numeric string. The product information symbol may be a barcode or the like in which information that recognizes the product information (such as a product information ID) is encoded, or a character string symbol that represents information that recognizes the product information.

[0022] Furthermore, for example, the recognition unit 2030 identifies the product by performing object recognition on the image of the product itself shown in the product image.

[0023] <Actions and Effects> The commodity registration device 2000 of this embodiment has multiple imaging units 2020. Therefore, if a commodity is imaged by any of the imaging units 2020, there is a possibility that the recognition unit 2030 can recognize the commodity. For example, suppose that an operator moves a commodity through the commodity passage area, but the commodity is not positioned within the imaging range of a certain imaging unit 2020. In this case, if the commodity registration device 2000 does not have another imaging unit 2020, the commodity will not be recognized by the recognition unit 2030 and will not be registered as a payment target. Therefore, the operator must redo the operation of having the imaging unit 2020 read the commodity information. On the other hand, if the commodity registration device 2000 has another imaging unit 2020, there is a possibility that the commodity will be positioned within the imaging range of the other imaging unit 2020, and therefore the recognition unit 2030 may be able to recognize the commodity.

[0024] As described above, product recognition using a product image is performed by analyzing a product information symbol or identifying the product through object recognition. Here, in order to analyze a product information symbol, the product information symbol must appear in the product image. Furthermore, in order to identify a product through object recognition, a characteristic part of the product must appear in the product image. For example, when identifying a type of canned coffee through object recognition, a product image showing only the bottom of the can is insufficient to identify the type; a product image showing a label or the like is required. According to the product registration device 2000 of this embodiment, if a product information symbol or a characteristic part of the product is captured by any of the imaging units 2020, the recognition unit 2030 can recognize the product.

[0025] Therefore, according to the commodity registration device 2000 of this embodiment, the probability that the commodity will be recognized by the recognition unit 2030 is higher than when only one imaging unit 2020 is provided. Therefore, the probability that the commodity will be registered as a payment target is higher, and the efficiency of the work of having the commodity registration device 2000 recognize the commodity is improved.

[0026] The product registration device 2000 of this embodiment will be described in further detail below.

[0027] <Example of Hardware Configuration of Merchandise Registration Device 2000> The product registration device 2000 may be realized by hardware alone (e.g., a hardwired electronic circuit, etc.), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.).

[0028] The merchandise registration device 2000 is implemented using a dedicated terminal such as a cash register terminal, for example. However, the merchandise registration device 2000 may be implemented using various general-purpose computers such as a PC (Personal Computer) or a server machine, instead of such a dedicated terminal.

[0029] FIG. 3 is a block diagram illustrating a hardware configuration of a computer 1000 used to implement the product registration device 2000. The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage 1080, and an input / output interface 1100. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage 1080, and the input / output interface 1100 to transmit and receive data to and from each other. However, the method of interconnecting the processor 1040 and other components is not limited to bus connection. The processor 1040 is a processing unit such as a central processing unit (CPU) or a graphics processing unit (GPU). The memory 1060 is a memory such as a random access memory (RAM) or a read-only memory (ROM). The storage 1080 is a storage device such as a hard disk, a solid state drive (SSD), or a memory card. The storage 1080 may also be a memory such as a RAM or a ROM. The storage 1080 stores various data and programs.

[0030] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. The computer 1000 is connected to a keyboard, a mouse, a display, or the like via the input / output interface 1100.

[0031] <<Hardware configuration of the imaging unit 2020>> For example, the imaging unit 2020 is realized by a camera 90 having an imaging element. Alternatively, for example, the imaging unit 2020 may be realized by a barcode reader. In this case, the imaging unit 2020 irradiates light onto the product and receives the reflected light with a light receiving element, thereby generating data representing the barcode pattern. The product image also includes the data representing this barcode pattern.

[0032] <Arrangement of the imaging unit 2020> The number of imaging units 2020 is any number equal to or greater than two. Specific examples of the arrangement of imaging units 2020 are shown below for cases where the number of imaging units 2020 is two, four, and eight. Note that the distance between imaging units 2020 in each arrangement is arbitrary. For example, this distance is 5 cm.

[0033] <<Placement 1>> Fig. 4 is a first perspective view illustrating an example of the arrangement of two imaging units 2020. In Fig. 4, the x direction is the direction in which the operator moves the product, the y direction is the depth direction of the table 20, and the z direction is the vertical direction.

[0034] Fig. 5 is a plan view corresponding to the arrangement in Fig. 4. Fig. 5(a) is a yz plan view of Fig. 4, and Fig. 5(b) is an xy plan view of Fig. 4. An imaging range 30 represents the imaging range of the imaging unit 2020. An imaging direction 40 is a direction that starts from the imaging unit 2020 and passes through the center of the imaging range 30 of the imaging unit 2020. Hereinafter, the imaging direction 40 of the imaging unit 2020 will also be referred to as the "orientation of the imaging unit 2020."

[0035] As shown in FIG. 5(a), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are different. Specifically, the position of the imaging unit 2020-1 in the y direction is one end of the stage 20 in the y direction, and the position of the imaging unit 2020-2 in the y direction is the other end of the stage 20 in the y direction. On the other hand, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the z direction are the same. Both of these are installed on the stage 20. Furthermore, as shown in FIG. 5(b), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the x direction are the same.

[0036] As shown in Fig. 5(a), the orientation of the imaging unit 2020-1 and the imaging unit 2020-2 is a direction in which the commodity passage area 10 is viewed from diagonally below in the yz plane. Note that, as shown in Fig. 5(b), it is preferable that the imaging ranges 30 of the imaging unit 2020-1 and the imaging unit 2020-2 partially overlap each other.

[0037] This arrangement allows the product to be imaged from multiple different directions in the y direction, which increases the probability that the product information symbol or a characteristic part of the product will appear in the product image compared to when the product is imaged from only one direction, thereby increasing the probability that the recognition unit 2030 will be able to recognize the product.

[0038] In the arrangements of FIGS. 4 and 5, the orientation of each imaging unit 2020 may be in the z direction.

[0039] <<Placement 2>> 6 is a second perspective view illustrating the arrangement of the two imaging units 2020. Here, the holding unit 2040 is a member for holding the imaging units 2020.

[0040] FIG. 7 is a plan view corresponding to the arrangement in FIG. 6. FIG. 7(a) is a yz plan view of FIG. 6, and FIG. 7(b) is an xy plan view of FIG. 6. As FIG. 7(a) shows, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are different, and their positional relationship is the same as in FIG. 4. Furthermore, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the z direction are the same, but are different from those in FIG. 4. Specifically, the imaging unit 2020-1 and the imaging unit 2020-2 are located above the commodity passage area 10. As FIG. 7(b) shows, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the x direction are the same.

[0041] As shown in Fig. 7(a), the orientation of the imaging unit 2020-1 and the imaging unit 2020-2 is a direction looking diagonally down from above in the yz plane view at the commodity passage area 10. Note that, as shown in Fig. 7(b), it is preferable that the imaging ranges 30 of the imaging unit 2020-1 and the imaging unit 2020-2 partially overlap.

[0042] This arrangement allows the product to be imaged from multiple different directions in the y direction, which increases the probability that the product information symbol or a characteristic part of the product will appear in the product image compared to when the product is imaged from only one direction, thereby increasing the probability that the recognition unit 2030 will be able to recognize the product.

[0043] Note that the orientation of each imaging unit 2020 in the arrangements of FIGS. 6 and 7 may be in the -z direction.

[0044] <<Placement 3>> FIG. 8 is a third perspective view illustrating the arrangement of two imaging units 2020. FIG. 9 is a plan view corresponding to the arrangement of FIG. 8. FIG. 9(a) is a yz plan view of FIG. 8, and FIG. 9(b) is an xy plan view of FIG. 8. As shown in FIG. 9(a), the imaging units 2020-1 and 2020-2 are positioned differently in the y direction, and their positional relationship is the same as that in FIG. 4. Furthermore, the imaging units 2020-1 and 2020-2 are positioned differently in the z direction. Specifically, the imaging unit 2020-1 is installed on the platform 20, and the imaging unit 2020-2 is held by the holding unit 2040 and positioned above the commodity passage area 10. As shown in FIG. 9(b), the imaging units 2020-1 and 2020-2 are positioned differently in the x direction.

[0045] As shown in Fig. 9(a), the imaging units 2020-1 and 2020-2 are oriented opposite each other across the commodity passage area 10 in the yz plane. As shown in Fig. 9(b), the imaging units 2020-1 and 2020-2 are also oriented opposite each other in the xy plane. Note that it is preferable that the imaging ranges 30 of the imaging units 2020-1 and 2020-2 partially overlap.

[0046] With this arrangement, the product is imaged from a plurality of different directions in the y direction. Therefore, compared to when the product is imaged from only one direction, the probability that the product information symbol or a characteristic part of the product will appear in the product image is higher, and the recognition unit 2030 is more likely to recognize the product. Furthermore, since the product is imaged from both above and below, the probability that the product information symbol or a characteristic part of the product will appear in the product image is higher compared to when the product is imaged only from above or only from below, and the recognition unit 2030 is more likely to recognize the product.

[0047] In the arrangements of FIGS. 8 and 9, the imaging unit 2020-1 may be oriented in the −z direction, and the imaging unit 2020-2 may be oriented in the z direction.

[0048] <<Placement 4>> FIG. 10 is a first perspective view illustrating an arrangement of four imaging units 2020. FIG. 11 is a plan view corresponding to the arrangement of FIG. 10. FIG. 11(a) is a yz plan view of FIG. 10, and FIG. 11(b) is an xy plan view of FIG. 10. As shown in FIG. 11(a), in the y direction, the imaging units 2020-1 and 2020-2 are positioned at the same position, and the imaging units 2020-3 and 2020-4 are positioned at the same position. Specifically, the positions of the imaging units 2020-1 and 2020-2 in the y direction are one end of the stage 20, and the positions of the imaging units 2020-3 and 2020-4 in the y direction are the other end of the stage 20. In the z direction, the positions of the imaging units 2020-1 and 2020-4 are the same, and the positions of the imaging units 2020-2 and 2020-3 are the same. Specifically, the imaging units 2020-1 and 2020-4 are held by a holder 2040 and positioned above the commodity passage area 10, and the imaging units 2020-2 and 2020-3 are installed on a platform 20. As shown in FIG. 11(b), the positions of the imaging units 2020 in the x direction are the same.

[0049] The imaging directions 40 of the imaging units 2020-1 and 2020-3 face each other. The imaging directions 40 of the imaging units 2020-2 and 2020-4 face each other. Note that it is preferable that the imaging directions 40 of the imaging units 2020-1 to 2020-4 face the center points of the imaging units 2020-1 to 2020-4, respectively. It is also preferable that the imaging ranges 30 of the imaging units 2020 partially overlap each other.

[0050] In this way, the four imaging units 2020 each capture an image of a product from a different direction, which reduces blind spots of the imaging units 2020 compared to when there are two imaging units 2020. Therefore, compared to when there are two imaging units 2020, there is a higher probability that the product information symbol or a characteristic part of the product will appear in the product image, which increases the probability that the recognition unit 2030 will be able to recognize the product.

[0051] Note that the orientation of each imaging unit 2020 in the arrangement shown in Fig. 8 and Fig. 9 is not limited to the orientation shown in Fig. 8 and Fig. 9. For example, the imaging unit 2020-1 and the imaging unit 2020-4 may face the z direction, and the imaging unit 2020-2 and the imaging unit 2020-3 may face the -z direction.

[0052] <<Placement 5>> FIG. 12 is a second perspective view illustrating an arrangement of four imaging units 2020. FIG. 13 is a plan view corresponding to the arrangement of FIG. 12. FIG. 13(a) is a yz plan view of FIG. 12, and FIG. 13(b) is an xy plan view of FIG. 12. As shown in FIG. 13(a), the positional relationship between these in the y direction and the z direction is the same as in FIG. 11(a). On the other hand, as shown in FIG. 13(b), the positional relationship between these in the x direction is different from that in FIG. 11(b). Specifically, the positions of imaging unit 2020-1 and imaging unit 2020-4 in the x direction are different from that in FIG. 11(b).

[0053] The relationship between the imaging directions 40 of the imaging units 2020 is the same as that in FIGS. It is also preferable that the imaging ranges 30 of the imaging units 2020 partially overlap each other.

[0054] 12 and 13 differ from the arrangements in Figures 10 and 11 in that the product is imaged from different directions in the x direction. Therefore, compared to the arrangements in Figures 10 and 11, there is a higher probability that the product information symbol or a characteristic part of the product will appear in the product image, and therefore there is a higher probability that the recognition unit 2030 will be able to recognize the product.

[0055] Note that the orientation of each imaging unit 2020 in the arrangements of Fig. 12 and Fig. 13 is not limited to the orientations shown in Fig. 12 and 13. For example, imaging unit 2020-1 and imaging unit 2020-2 may be oriented in corresponding directions, and imaging unit 2020-3 and imaging unit 2020-4 may be oriented in opposing directions.

[0056] <<Placement 6>> FIG. 14 is a diagram illustrating an example of the arrangement of eight imaging units 2020. FIG. 15 is a plan view corresponding to the arrangement of FIG. 14. FIG. 15(a) is a yz plan view of FIG. 14, and FIG. 15(b) is an xy plan view of FIG. 14. As these figures show, the positional relationship between imaging units 2020-1 to 2020-4 and the positional relationship between imaging units 2020-5 to 2020-8 are the same as the positional relationships in the arrangements of FIGS. 10 and 11, respectively. However, as shown in FIG. 15(b), the positions of imaging units 2020-1 to 2020-4 and imaging units 2020-5 to 2020-8 in the x direction are different.

[0057] The imaging directions 40 of the imaging units 2020-1 and 2020-7, the imaging directions 40 of the imaging units 2020-2 and 2020-8, the imaging directions 40 of the imaging units 2020-3 and 2020-5, and the imaging directions 40 of the imaging units 2020-4 and 2020-6 are opposed to each other. Note that it is preferable that the imaging directions 40 of each imaging unit 2020 all face the center positions of all the imaging units 2020. It is also preferable that the imaging ranges 30 of each imaging unit 2020 partially overlap each other.

[0058] 14 and 15, the blind spots of the imaging unit 2020 are reduced compared to when the number of imaging units 2020 is two or four. Therefore, the probability that the product information symbol or a characteristic part of the product will appear in the product image increases, and the probability that the recognition unit 2030 will be able to recognize the product increases.

[0059] <<About the holding unit 2040>> The holder 2040 is preferably made of a material that transmits visible light. For example, this material is a rod, pillar, or plate made of transparent plastic, glass, or the like. By making the holder 2040 transmit visible light, it is possible to prevent the holder 2040 from blocking light from reaching the product. As a result, it is possible to prevent a decrease in the accuracy of product recognition by the recognition unit 2030.

[0060] <Processing flow> 16 is a flowchart illustrating the flow of processing executed by the commodity registration device 2000 of embodiment 1. The imaging unit 2020 generates a commodity image (S102). The recognition unit 2030 recognizes the commodity using the commodity image (S104).

[0061] <Details of the processing performed by the imaging unit 2020> The image capturing unit 2020 generates a product image (S102). The image capturing unit 2020 may capture a still image or a video. In the latter case, the operator image is each frame constituting the video.

[0062] The timing at which the imaging unit 2020 captures an image varies. For example, the imaging unit 2020 captures an image at the timing when the operator causes the merchandise registration device 2000 to recognize a merchandise, and before and after that timing. For example, an infrared sensor that detects people is provided near the merchandise registration device 2000. The merchandise registration device 2000 can determine that an operator is near the merchandise registration device 2000 by receiving a notification from the infrared sensor. Therefore, for example, the imaging unit 2020 captures an image from the time when the infrared sensor detects that an operator is near the merchandise registration device 2000 until the infrared sensor no longer detects the operator.

[0063] Furthermore, for example, the imaging unit 2020 may periodically and repeatedly capture images, for example, every 1 / 30 seconds, which is the same as the frame rate of a typical video.

[0064] <Details of the process performed by the recognition unit 2030> The recognition unit 2030 recognizes the product using the product images generated by the imaging unit 2020 (S104). For example, the recognition unit 2030 identifies the product by detecting a product information symbol appearing in each product image generated by each imaging unit 2020 and analyzing the detected product information symbol. Alternatively, for example, the recognition unit 2030 identifies the product by performing object recognition on the product itself appearing in the product image generated by each imaging unit 2020. Then, if the product can be identified from any of the product images, the identified product is registered as a product to be checked out.

[0065] Here, the identification results obtained for each of the multiple product images may indicate different products. For example, suppose that an identification result using a product image generated by one imaging unit 2020 indicates that "product is X," while an identification result using a product image generated by another imaging unit 2020 indicates that "product is Y." In this case, the recognition unit 2030 identifies the product using the most likely result. For example, the recognition unit 2030 may adopt the result with the most product images that result in the same identification result. For example, suppose that the analysis results of three product images indicate that "product is X," while the analysis results of five other product images indicate that "product is Y." In this case, the recognition unit 2030 adopts the result with the most product images, "product is Y," and registers product Y as the product to be checked out. However, the recognition unit 2030 may issue a warning if it is unable to uniquely identify the product.

[0066] [Embodiment 2] The commodity registration device 2000 of the second embodiment is shown in Fig. 1 in the same manner as the commodity registration device 2000 of the first embodiment. The commodity registration device 2000 of the second embodiment is the same as the commodity registration device 2000 of the first embodiment except that the arrangement of the imaging unit 2020 is different.

[0067] In the commodity registration device 2000 of the second embodiment, at least two of the multiple image capturing units 2020 are arranged so as to capture images of commodities that are in different directions from each other. The arrangement of the image capturing units 2020 will be specifically described below.

[0068] <Placement 1> 17 is a perspective view illustrating an example of the arrangement of three imaging units 2020. The x direction, y direction, and z direction are the direction in which the operator moves the product, the depth direction of the platform 20, and the vertical direction, respectively. The imaging range 30 is the imaging range of the imaging unit 2020. The imaging direction 40 is a direction that starts from the imaging unit 2020 and passes through the center of the imaging range 30 of the imaging unit 2020.

[0069] 17, the imaging directions 40 of the imaging units 2020 are different from one another. The imaging units 2020 are housed in the same housing 100.

[0070] Fig. 18 is a plan view corresponding to the arrangement of Fig. 17. Fig. 18(a) is a yz plan view of Fig. 17, and Fig. 18(b) is an xy plan view of Fig. 17. As shown in Fig. 18(b), each imaging unit 2020 captures an image of a commodity 50 that is in a different direction from each other.

[0071] By arranging the imaging units 2020 in this manner, images of the product are captured from multiple different directions, increasing the probability that the product information symbol or a characteristic part of the product will appear in the product image, and increasing the probability that the recognition unit 2030 will be able to recognize the product. Furthermore, as shown in the arrangements of Figures 17 and 18, multiple imaging units 2020 can be arranged close to each other, allowing the multiple imaging units 2020 to be housed in the same housing. This makes it easy to install the imaging units 2020.

[0072] <Placement 2> FIG. 19 is a perspective view illustrating an example of the arrangement of eight imaging units 2020. In FIG. 19, imaging units 2020-1 to 2020-3 are housed in the same housing 100-1. Similarly, imaging units 2020-4 to 2020-6 are housed in the same housing 100-2. Housings 100-1 and 100-2 are installed so as to face each other. Imaging unit 2020-7 is installed on platform 20. Meanwhile, imaging unit 2020-8 is installed above the commodity passage area 10. Although not shown in the figure, imaging unit 2020-8 is held by holding unit 2040.

[0073] FIG. 20 is a plan view corresponding to the arrangement of FIG. 19. FIG. 20(a) is a yz plan view of FIG. 19, and FIG. 20(b) is an xy plan view of FIG. 19. As shown in FIG. 20(b), imaging units 2020-1 to 2020-3 capture images of merchandise 50 in different directions, similar to the arrangement of FIG. 17. Imaging units 2020-4 to 2020-6 also capture images of merchandise 50 in different directions. Imaging units 2020-2 and 2020-5, and imaging units 2020-3 and 2020-6, respectively, capture images of merchandise 50 in the same position from different directions. Imaging units 2020-1, 2020-4, 2020-7, and 2020-8 capture images of merchandise 50 in the same position from different directions.

[0074] According to the arrangements of Figures 19 and 20, more imaging units 2020 can be used to capture images of the product 50 from various directions than in the case of Figures 17 and 18, which increases the probability that the product information symbol or a characteristic part of the product will appear in the product image, and increases the probability that the recognition unit 2030 will be able to recognize the product.

[0075] The hardware configuration of the commodity registration device 2000 of the second embodiment is the same as that of the commodity registration device 2000 of the first embodiment, except for the arrangement of the imaging unit 2020.

[0076] Furthermore, the processes executed by the functional components of the merchandise registration device 2000 of the second embodiment are similar to the processes executed by the functional components of the merchandise registration device 2000 of the first embodiment.

[0077] <Actions and Effects> According to the commodity registration device 2000 of this embodiment, since the multiple imaging units 2020 capture images of commodities from different directions, the probability that the commodity information symbol or a characteristic part of the commodity will appear in the commodity image is higher than when there is only one imaging unit 2020. Therefore, the probability that the recognition unit 2030 can recognize the commodity is higher. Furthermore, in the commodity registration device 2000 of this embodiment, multiple commodity registration devices 2000 are installed in the same or nearby locations, and these imaging units 2020 capture images of commodities that are in different directions. Therefore, these imaging units 2020 can be housed in a common housing, making it easier to install the imaging units 2020.

[0078] [Embodiment 3] Fig. 21 is a block diagram illustrating a product registration device 2000 according to embodiment 3. In Fig. 21, each block represents a functional configuration rather than a hardware configuration.

[0079] The commodity registration device 2000 of the third embodiment has a control unit 2060. The control unit 2060 captures an image of the operator's actions and generates an image. Hereinafter, the image generated by the control unit 2060 is referred to as an operator image. The control unit 2060 then controls the imaging unit 2020 using the operator image.

[0080] <Hardware Configuration of Merchandise Registration Device 2000> The hardware configuration of the merchandise registration device 2000 of the third embodiment is shown in FIG. 3, similar to the hardware configuration of the merchandise registration device 2000 of the first embodiment.

[0081] The control unit 2060 has an imaging element for generating an image of the operator. For example, the control unit 2060 is implemented using a camera having an imaging element. FIG. 22 is a diagram specifically illustrating a merchandise registration device 2000 of the third embodiment. In the merchandise registration device 2000 of FIG. 22, a camera 110 for implementing the control unit 2060 is installed on the holder 2040. However, the installation position of the camera 110 is not limited to the position shown in FIG. 22.

[0082] The control unit 2060 may have one or more cameras.

[0083] Here, the resolution of the imaging element of the control unit 2060 may be lower than the resolution of the imaging element of the imaging unit 2020. This is because, as will be described later, the resolution of the operator image, which only needs to be usable for detecting the position of the product and the product information symbol, is lower than the resolution of the product image, which must be usable for identifying the product. By lowering the resolution of the imaging element of the control unit 2060 in this way, the manufacturing cost of the product registration device 2000 can be reduced. However, the resolution of the imaging element of the control unit 2060 may be equal to or higher than the resolution of the imaging element of the imaging unit 2020.

[0084] The storage 1080 of the third embodiment further stores program modules for realizing the respective functional components of the third embodiment. The processor 1040 then executes these program modules to realize the functions of the respective functional components of the fourth embodiment.

[0085] <Processing flow> 23 is a flowchart illustrating the flow of processing executed by the commodity registration device 2000 of embodiment 3. The control unit 2060 generates an operator image (S202). The control unit 2060 controls the imaging unit 2020 based on the operator image (S204). The imaging unit 2020 captures an image of the commodity under the control of the control unit 2060 and generates a commodity image (S206). The recognition unit 2030 recognizes the commodity using the generated commodity image (S208).

[0086] <Details of the processing performed by the control unit 2060> The control unit 2060 generates an operator image (S202). The control unit 2060 may capture a still image or may capture a moving image. In the latter case, the operator image is each frame that constitutes the moving image.

[0087] The control unit 2060 may capture images at various times, which are similar to the timing at which the image capturing unit 2020 described in the first embodiment captures images.

[0088] Furthermore, the control unit 2060 uses the operator image to control the image capturing unit 2020 (S204). There are various specific methods by which the control unit 2060 controls the image capturing unit 2020. Examples of such methods will be described below.

[0089] <<Control method 1>> The control unit 2060 controls the timing at which each imaging unit 2020 captures an image of a product. First, the control unit 2060 uses the operator image to detect the product and calculate the direction and speed of movement of the product. Here, technology such as object recognition can be used to detect the product from the image. The control unit 2060 also calculates the direction and speed of movement of the product based on changes in the position of the product captured in multiple operator images.

[0090] Then, the control unit 2060 calculates, for each imaging unit 2020, the timing at which the product will be placed within the imaging range of the imaging unit 2020, using the imaging time of the operator image, the moving direction of the product, and the moving speed of the product. Then, the control unit 2060 causes each imaging unit 2020 to image the product at the timing calculated for each imaging unit 2020. In this way, each imaging unit 2020 can be operated at a timing at which the recognition unit 2030 is highly likely to recognize the product, thereby increasing the probability that the recognition unit 2030 will recognize the product.

[0091] Furthermore, the control unit 2060 may not cause the imaging unit 2020, which does not have a product located within its imaging range, to capture an image of the product. This can reduce the power consumption of the imaging unit 2020 and prevent deterioration of the imaging unit 2020.

[0092] Here, when the image of the product is captured by the imaging unit 2020, the product is held in the operator's hand, and therefore part of the surface of the product is covered by the operator's hand. Therefore, the image of the operator may show the product with part of the product missing.

[0093] In such a case, one possible method is to complement the portions not shown in the operator image to reproduce the entire product, and then perform control to determine the timing of capturing an image by the imaging unit 2020. However, it is preferable that the control unit 2060 does not perform such complementation, but instead determines the timing of capturing an image by the imaging unit 2020 using the area of ​​the product shown in the operator image. This is because there is a high probability that a portion covered by the operator's hand will not be captured in the product image generated by the imaging unit 2020, and therefore it is not appropriate to have the imaging unit 2020 capture an image of the product at a timing when such a portion is located.

[0094] Therefore, for example, the control unit 2060 determines the timing of capturing images for each imaging unit 2020 by regarding only the area of ​​the product that is shown in the operator image as representing the product. In this way, the control unit 2060 can cause the imaging units 2020 to capture images at a timing when the area of ​​the product shown in the product image (the part not covered by a hand, etc.) is located within the imaging range. Furthermore, the control unit 2060 can cause only the imaging units 2020 whose imaging range includes the area of ​​the product shown in the product image to capture images.

[0095] Note that the moving speed of the product may be set as a fixed value in advance, rather than being calculated using the operator image. For example, before starting operation of the product registration device 2000, the developer of the product registration device 2000 may conduct an operation test in which the product registration device 2000 recognizes products. In this way, the developer repeatedly measures the moving speed of the product when the product registration device 2000 recognizes the product, and determines the fixed value using the measurement results. In this case, the control unit 2060 performs the same control as described above, using the image capture time of the operator image, the moving direction of the product calculated from the operator image, and the moving speed of the product set as a fixed value in advance.

[0096] The direction of movement of the product may also be set as a fixed value in advance, for example, the long side direction of the platform 60.

[0097] The above-mentioned fixed values ​​may be preset in the control unit 2060, or may be stored in a storage unit accessible from the control unit 2060. In the latter case, the control unit 2060 reads out and uses the fixed values ​​from the storage unit.

[0098] Furthermore, if the positions and imaging directions of the cameras or the like included in the control unit 2060 and the position and recognition direction of the imaging unit 2020 are fixed, it is possible to determine in advance which imaging unit 2020 should capture an image when a product is detected at that position, in association with the position of the product in the operator image. Therefore, such a predetermined control method is stored in a storage unit accessible by the control unit 2060. In this case, when the control unit 2060 detects a product from the operator image, it controls the imaging unit 2020 by referring to the information stored in this storage unit.

[0099] <<Control method 2>> The control unit 2060 calculates the position, movement speed, and movement direction of the commodity information symbol, and controls the imaging unit 2020 based on these. For example, the control unit 2060 causes the imaging unit 2020 to capture an image at a timing when the commodity information symbol is located within the imaging range of the imaging unit 2020. Furthermore, for example, the control unit 2060 controls the imaging unit 2020 so that only the imaging units 2020 in which the commodity information symbol is located within the imaging range capture an image, and so that the imaging units 2020 in which the commodity information symbol is not located within the imaging range do not capture an image. Here, the method for calculating the position, movement speed, and movement direction of the commodity information symbol is the same as the method for calculating the position, movement speed, and movement direction of the commodity information symbol. In addition, the method of determining the timing for each imaging unit 2020 to take an image or having only some of the imaging units 2020 take an image based on the position, movement speed, and movement direction of the product information symbol is similar to the method of determining the timing for each imaging unit 2020 to take an image or having only some of the imaging units 2020 take an image based on the position, movement speed, and movement direction of the product.

[0100] Here, the commodity information symbol may not appear in the operator image. For example, when the control unit 2060 captures an image of the operator's movements from diagonally above, the commodity information symbol attached below the commodity may not appear in the image. Therefore, if the control unit 2060 detects a commodity from the operator image but cannot detect the commodity information symbol, the control unit 2060 may control the imaging units 2020 so that an imaging unit 2020 located in a blind spot of the camera of the control unit 2060 captures an image and the other imaging units 2020 do not capture an image. The imaging units 2020 located in a blind spot of the camera of the control unit 2060 can be known in advance from the position and imaging direction of the control unit 2060 and the arrangement of each imaging unit 2020.

[0101] This method of controlling the imaging unit 2020 based on the position of the commodity information symbol, etc., makes it possible to cause the imaging unit 2020 to capture an image at more appropriate timing than a method of controlling the imaging unit 2020 based on the position of the commodity, etc., and to cause only the imaging unit 2020 that can read the commodity information symbol to capture an image. In this case, the recognition unit 2030 recognizes the commodity by analyzing the commodity information symbol.

[0102] <Actions and Effects> According to this embodiment, the image capturing unit 2020 is controlled based on the operator image. Therefore, as described above, it is possible to increase the probability that the recognition unit 2030 can recognize the product, reduce the power consumption of the image capturing unit 2020, and prevent deterioration of the image capturing unit 2020.

[0103] [Embodiment 4] The merchandise registration device 2000 of the fourth embodiment is represented by Fig. 21 in the same manner as the merchandise registration device 2000 of the third embodiment. Except for the points described below, the merchandise registration device 2000 of the fourth embodiment is the same as the merchandise registration device 2000 of the third embodiment.

[0104] The control unit 2060 of the fourth embodiment determines one or more product images to be used for product recognition based on the operator image and the positions and orientations of the imaging units 2020. The recognition unit 2030 recognizes the product using the product images determined by the control unit 2060.

[0105] <Hardware configuration> The hardware configuration of the product registration device 2000 of the fourth embodiment is shown in Fig. 3, similar to the hardware configuration of the product registration device 2000 of the first embodiment. The storage 1080 of the fourth embodiment further stores program modules for realizing each functional component of the fourth embodiment. The processor 1040 executes each of these program modules to realize the function of each functional component of the fourth embodiment.

[0106] <Processing flow> 24 is a flowchart illustrating the flow of processing executed by the product registration device 2000 of embodiment 4. Each imaging unit 2020 generates a product image (S302). The control unit 2060 generates an operator image (S304). The control unit 2060 uses the operator image to determine a product image to be used for product recognition (S306). The recognition unit 2030 recognizes the product using the determined product image (S308).

[0107] <Details of the processing performed by the control unit 2060> The control unit 2060 determines the product image using the operator image (S306). To do so, the control unit 2060 calculates the timing at which the product is to be placed within the imaging range of each imaging unit 2020 in the same manner as the control unit 2060 in the third embodiment. However, in this embodiment, the method of using the calculated timing, etc., differs from that in the third embodiment. This will be described in detail below.

[0108] <<Control method 1>> For example, the control unit 2060 determines the timing at which a product is placed within the imaging range for each imaging unit 2020 using a method similar to control method 1 described in embodiment 3. Then, for each imaging unit 2020, the control unit 2060 identifies, from among the product images generated by that imaging unit 2020, a product image that was captured at the timing at which the product was placed within the imaging range. Then, the control unit 2060 determines that each identified product image is to be used for product recognition.

[0109] Furthermore, for example, the control unit 2060 identifies the imaging unit 2020 within whose imaging range the commodity is placed, using a method similar to control method 1 described in embodiment 3. Then, the control unit 2060 determines that the commodity image generated by the imaging unit 2020 within whose imaging range the commodity is placed is to be used for commodity recognition.

[0110] <<Control method 2>> For example, the control unit 2060 determines the timing at which the commodity information symbol is placed within the imaging range using a method similar to control method 2 described in embodiment 3. Then, for each imaging unit 2020, the control unit 2060 identifies the commodity image that was captured at the timing at which the commodity information symbol was placed within the imaging range from among the commodity images generated by the imaging unit 2020. Then, the control unit 2060 determines that each identified commodity image is to be used for commodity recognition.

[0111] Furthermore, for example, the control unit 2060 identifies the imaging unit 2020 whose commodity information symbol is arranged within the imaging range by a method similar to control method 2 described in embodiment 3. Then, the control unit 2060 determines that the commodity image generated by the imaging unit 2020 whose commodity information symbol is arranged within the imaging range is to be used for commodity recognition.

[0112] <Actions and Effects> According to this embodiment, the probability that the merchandise registration device 2000 can recognize the merchandise increases for the same reasons as in embodiment 3. Furthermore, according to this embodiment, the control unit 2060 does not need to control the imaging unit 2020. Therefore, the control of the imaging unit 2020 by the merchandise registration device 2000 becomes simple.

[0113] [Embodiment 5] The merchandise registration device 2000 of the fifth embodiment is shown in Fig. 21, similar to the merchandise registration device 2000 of the fourth embodiment. Except for the points described below, the merchandise registration device 2000 of the fifth embodiment is similar to the merchandise registration device 2000 of the fourth embodiment.

[0114] The control unit 2060 of the fourth embodiment calculates the timing at which the product or product information symbol is placed within the imaging range of each imaging unit 2020. However, since the timing calculated by the control unit 2060 is a predicted value, the timing at which the product or product information symbol is actually placed within the imaging range of each imaging unit 2020 may differ from this predicted value.

[0115] Furthermore, the control unit 2060 of the fourth embodiment calculates the moving direction of the product, etc., and predicts the imaging unit 2020 within whose imaging range the product or product information symbol will be placed. However, since the moving direction of the product may change, the product may not be placed within the imaging range of the imaging unit 2020 that was predicted to "place the product within the imaging range," or the product may be placed within the imaging range of the imaging unit 2020 that was predicted to "not place the product within the imaging range."

[0116] Therefore, the control unit 2060 of the fifth embodiment weights the product images captured by the imaging unit 2020. Then, the recognition unit 2030 recognizes the product using each weighted product image.

[0117] <Processing flow> Fig. 25 is a flowchart illustrating the flow of processing executed by the product registration device 2000 of embodiment 5. The processing performed in S302 and S304 is the same as that in Fig. 23. The control unit 2060 assigns weighting to each product image (S402). The recognition unit 2030 recognizes the product using the weighted product image (S404).

[0118] <Details of the processing performed by the control unit 2060> The control unit 2060 of the fifth embodiment assigns a weight to each product image using the operator image (S402). Specifically, the control unit 2060 assigns weights to the product images using the control methods described in the third and fourth embodiments. For example, the control unit 2060 assigns a higher weight to an imaging unit 2020 that is predicted to have a product or product information symbol placed within the imaging range than to an imaging unit 2020 that is predicted to have a product or product information symbol not placed within the imaging range. Furthermore, the control unit 2060 assigns a higher weight to a product image that is captured closer to the timing at which the product or product information symbol is predicted to be placed within the imaging range.

[0119] Here, the weight that the control unit 2060 assigns to the product image i based on whether or not the product or product information symbol is placed within the imaging range is ai, and the weight that the control unit 2060 assigns to the product image i based on the discrepancy between the timing when the product or product information symbol is placed within the imaging range and the time when the product image is captured is bi. In this case, for example, the control unit 2060 assigns a weight of ai*bi to the product image i. However, the control unit 2060 may assign only ai or bi to the product image i.

[0120] <Details of the process performed by the recognition unit 2030> The recognition unit 2030 recognizes products using the weighted product images (S404). Here, the recognition unit 2030 uses the weight assigned to the product image as an index of the likelihood of the product being recognized using that product image. Specifically, the recognition unit 2030 identifies the product shown in each product image that shows the same product. If the recognition unit 2030 is unable to uniquely identify a product, it uses the weight assigned to each product image as the likelihood of identification based on that product image, thereby uniquely identifying the product.

[0121] For example, among multiple product images generated for a certain product, some of the product images are identified as "product X" through object recognition or the like. On the other hand, other product images are identified as "product Y" through object recognition or the like. In this case, the recognition unit 2030 calculates the two probabilities of identification using the weights assigned to each product image, and determines the more probable result as the product identification result. For example, the recognition unit 2030 uses the weight of the product image identified as "product X" and the weight of the product image identified as "product Y" as the respective probabilities of identification. Then, the recognition unit 2030 determines the identification with the larger weight as the final identification result.

[0122] In the above example, it is assumed that there are multiple product images identified as "product X" and multiple product images identified as "product Y." In this case, the recognition unit 2030 compares the total weight value of the product images identified as "product X" with the total weight value of the product images identified as "product Y," and determines the identification with the larger total weight value as the final identification result.

[0123] <Actions and Effects> According to the commodity registration device 2000 of this embodiment, it is possible to uniquely identify a commodity with high accuracy by weighting each commodity image using a predicted value of the timing at which the commodity will be placed within the imaging range of the imaging unit 2020. Therefore, the commodity registration device 2000 can recognize the commodity with high accuracy.

[0124] [Embodiment 6] Fig. 26 is a block diagram illustrating an adjustment system 4000 according to embodiment 6. In Fig. 1, each block represents a configuration in functional units, rather than a configuration in hardware units.

[0125] The settlement system 4000 includes any one of the merchandise registration devices 2000 according to the first to fifth embodiments, and a settlement device 3000. The settlement device 3000 is a device that performs settlement processing for merchandise recognized by the recognition unit 2030.

[0126] The product registration device 2000 generates settlement information to be used for settling products that have been registered as items subject to settlement through recognition by the recognition unit 2030. Here, the settlement items in one settlement process may include multiple products. For example, after receiving an operation to start the registration process for the items subject to settlement, the product registration device 2000 registers one or more products recognized by the recognition unit 2030 as items subject to settlement in one settlement process until receiving an operation to end the registration process for the items subject to settlement. The settlement information for a certain settlement process indicates, for example, the ID of each product registered as a target for that settlement process. The settlement information may also further indicate, for example, a transaction number, the price of each product, and the total price.

[0127] The settlement device 3000 is used to process the settlement of products registered as items to be settled. Specifically, the settlement device 3000 performs processes such as presenting the total amount, accepting the payment, counting the payment, returning the change, and issuing a receipt.

[0128] The settlement device 3000 may be provided integrally with or separately from the merchandise registration device 2000. When the settlement device 3000 is provided integrally with the merchandise registration device 2000, the computer 1000 has not only the function of operating as the merchandise registration device 2000 but also the function of operating as the settlement device 3000.

[0129] When the settlement device 3000 is provided separately from the product registration device 2000, the settlement device 3000 may be implemented as a dedicated terminal such as a cash register terminal, or as a general-purpose computer. The hardware configuration of the computer that realizes the settlement device 3000 is, for example, similar to the hardware configuration of the computer 1000 in Fig. 3. Furthermore, the input / output interface of the computer that realizes the settlement device 3000 is connected to a device for a customer to insert payment, a device for returning change, a device for issuing receipts, and the like.

[0130] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0131] Below, examples of reference forms are given. 1. The system has a plurality of imaging means at different positions for capturing images of products and generating product images, The product registration device has a recognition means for recognizing a product using an image of the product or an image of a product information symbol attached to the product, which is a symbol for identifying the product, that appears in the product image generated by each of the imaging means. 2. The product registration device described in 1., wherein the first imaging means and the second imaging means capture images of the product from different directions. 3. A product registration device as described in 1. or 2., wherein the first imaging means and the second imaging means are provided in positions facing each other across a product passing area through which products pass. 4. A plurality of imaging means for capturing images of products and generating product images; a recognition means for recognizing a product using an image of the product or an image of a product information symbol attached to the product, which is a symbol for identifying the product, that appears in the product image generated by each of the imaging means; A merchandise registration device, wherein each of the imaging means images merchandise in different directions. 5. A product registration device as described in any one of 1. to 4., wherein each of the imaging means has an imaging range that includes part or all of the product passing area through which the product passes. 6. A holding means for holding the imaging means and for transmitting visible light therethrough is provided. The product registration device according to any one of 1. to 5., wherein the visible light that has passed through the holding means passes through a product passage area through which the product passes. 7. A product registration device described in any one of 1. to 6., which has a control means for capturing an image of an operator by capturing an image of the operator, and using the image of the operator to cause some of the multiple imaging means to capture an image of the product. 8. A control means for capturing an image of the operator's movements to generate an operator image, and using the operator image to determine a product image to be used for product recognition from among the product images captured by each imaging means; The product registration device according to any one of 1. to 6., wherein the recognition means recognizes the product using the product image determined by the control means. 9. A control means for capturing an image of an operator to generate an operator image and determining a commodity image to be used for commodity recognition using the operator image; The product registration device according to any one of 1. to 6., wherein the recognition means recognizes products using the weighted product images. 10. A program that causes a computer to operate as the product registration device described in any one of 1. to 9. 11. A control method executed by a computer, comprising: an imaging step in which imaging means provided at different positions capture images of the commodity to generate a commodity image; A control method comprising: a recognition step of recognizing the product using an image of the product or an image of a product information symbol attached to the product, which is a symbol that identifies the product, that appears in the product image generated by each of the imaging means. 12. The control method described in 11, wherein the first imaging means and the second imaging means capture images of the product from different directions. 13. A control method described in 11. or 12., wherein the first imaging means and the second imaging means are provided in positions facing each other across a product passing area through which products pass. 14. A control method executed by a computer, comprising: an imaging step in which a plurality of imaging means capture images of products to generate product images; a recognition step of recognizing the product using an image of the product or an image of a product information symbol attached to the product, which is a symbol for identifying the product, that appears in the product image generated by each of the imaging means; A control method in which each of the imaging means images a product in a different direction. 15. A control method according to any one of claims 11 to 14, wherein each of the imaging means has an imaging range that includes part or all of a product passing area through which the product passes. 16. A holding means for holding the imaging means and for transmitting visible light therethrough, A control method according to any one of claims 11 to 15, wherein the visible light that has passed through the holding means passes through a product passage area through which the product passes. 17. A control method described in any one of 11 to 16, which includes a control step of capturing an image of an operator by capturing an image of the operator, and using the image of the operator to cause some of the multiple imaging means to capture an image of the commodity. 18. A control step of capturing an image of the operator's movements to generate an operator image, and using the operator image to determine a product image to be used for product recognition from among the product images captured by each imaging means; The control method according to any one of 11 to 16, wherein the recognition step recognizes the product using the product image determined by the control step. 19. A control step of capturing an image of an operator to generate an operator image and using the operator image to determine a commodity image to be used for commodity recognition, The control method according to any one of 11 to 16, wherein the recognition step recognizes the product using each of the weighted product images. 20. A settlement system having a product registration device described in any one of 1. to 9. and a settlement device that performs settlement processing for products recognized by the recognition means of the product registration device. [Explanation of symbols]

[0132] 10 Product Passing Area 20 units 30 Imaging range 40 Imaging direction 50 items 60 units 90 Camera 100 cabinets 110 Camera 1000 calculator 1020 Bus 1040 processor 1060 memory 1080 Storage 1100 Input / Output Interface 2000 Product Registration Device 2020 Imaging Unit 2030 Recognition part 2040 Holding part 2060 Control Unit

Claims

1. a recognition means for recognizing an object using a plurality of object images captured by the imaging means; a determination means for determining which recognition result to output based on the number of recognitions of the same recognition result when at least one of the recognition results of the object based on each of the plurality of object images is different from at least one other recognition result; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the determining means determines the recognition result with the largest number of identically recognized characters to be the recognition result to be output.

3. The information processing apparatus according to claim 1 , further comprising a notification unit that notifies the user when at least one of the recognition results is different from at least one of the other recognition results.

4. 4. The information processing device according to claim 1, further comprising a control means for determining a timing at which the object captured in the operator image is positioned within an imaging range of the imaging means based on a change in the position of the object captured in a plurality of time-series operator images of the operator, and for causing the imaging means to capture an image at the determined timing and generate the object image.

5. 4. The information processing device according to claim 1, further comprising a control means for determining a timing at which the object captured in the operator image is positioned within the imaging range of the imaging means based on a change in the position of the object captured in a plurality of time-series operator images of the operator, and determining the object image captured and generated at the determined timing as the object image to be used for recognizing the object.

6. The computer Recognizing an object using a plurality of object images captured by an imaging means; A control method for determining which recognition result to output when at least one of the recognition results of the object based on each of the multiple object images is different from at least one other recognition result, based on the number of identical recognition results in each of the recognition results.

7. The control method according to claim 6 , wherein the recognition result with the largest number of recognized identical characters is determined as the recognition result to be output.

8. 8. The control method according to claim 6, further comprising the step of: notifying when at least one of the recognition results differs from at least one other of the recognition results.

9. 9. The control method according to claim 6, further comprising: identifying a timing at which the object captured in the operator image is positioned within an imaging range of the imaging means based on a change in position of the object captured in a plurality of time-series operator images of the operator; and causing the imaging means to capture an image at the identified timing and generate the object image.

10. 9. A control method according to claim 6, wherein a timing at which the object captured in the operator image is placed within the imaging range of the imaging means is identified based on a change in the position of the object captured in a plurality of time-series operator images of the operator, and the object image captured and generated at the identified timing is determined to be the object image to be used for recognizing the object.

Images