Information Processing Apparatus and Control Method

Abstract

To solve a problem that arises when using multiple images generated by multiple imaging means to identify commodities.SOLUTION: An information processor includes an acquisition unit, a recognition unit 2030, and a notification unit. The acquisition unit acquires an image of a commodity to be settled that is captured by a plurality of imaging units 2020. The recognition unit 2030 recognizes the commodities based on the images. The notification unit performs notification when recognition results based on the images captured by each of the imaging units 2020 are different.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a control method.

Background Art

[0002] In stores such as supermarkets, cash registers operated by an operator himself / herself have started to be used. The operator registers the purchased products by making a cash register recognize barcodes or the like attached to the products to be purchased. Then, the operator purchases the registered products by inserting the price displayed on the screen into the cash register.

[0003] As a document disclosing a technique related to such a cash register, for example, there is Patent Document 1. Patent Document 1 discloses a technique that has one scanner and one camera, and determines the identity or difference between a scanned product and an imaged product by using the result of scanning the product and the result of imaging the product by the camera.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In order to improve the recognition accuracy of products, it is conceivable to identify products by using a plurality of images generated by a plurality of imaging means.

[0006] The present invention has been made in view of the above problems. An object of the present invention is to solve problems that occur when identifying a product by using a plurality of images generated by a plurality of imaging means.

Means for Solving the Problems

[0007] The first information processing apparatus of the present invention includes an acquisition means for acquiring an image of a product to be settled, which is captured by a plurality of imaging means, a recognition means for recognizing the product based on the image, and a notification means for performing a notification when recognition results based on images captured by the respective imaging means are different.

[0008] The second information processing apparatus of the present invention includes an acquisition means for acquiring an image of a product to be settled, which is captured by a first imaging means, a second imaging means, and a third imaging means, and a recognition means for recognizing the product based on the image, wherein the second imaging means and the third imaging means are arranged at positions different from those of the first imaging means in the vertical direction and the horizontal direction.

[0009] The third information processing apparatus of the present invention includes a first imaging means for imaging a product to be settled, a second imaging means and a third imaging means provided at positions different from those of the first imaging means in the vertical direction and the horizontal direction and for imaging the product, and an output means for outputting information regarding the product recognized based on images captured by the first imaging means, the second imaging means, and the third imaging means.

[0010] The first control method of the present invention is such that a computer acquires an image of a product to be settled, which is captured by a plurality of imaging means, recognizes the product based on the image, and performs a notification when recognition results based on images captured by the respective imaging means are different.

[0011] The second control method of the present invention is such that a computer acquires an image of a product to be settled, which is captured by a first imaging means, a second imaging means, and a third imaging means, recognizes the product based on the image, and the second imaging means and the third imaging means are arranged at positions different from those of the first imaging means in the vertical direction and the horizontal direction.

[0012] The third control method of the present invention is that a computer captures an image of a product to be settled using a first imaging means, and captures the product using a second imaging means and a third imaging means provided at positions different from the first imaging means in the vertical and horizontal directions, and outputs information regarding the product recognized based on the images captured by the first imaging means, the second imaging means, and the third imaging means.

Advantages of the Invention

[0013] According to the present invention, it is possible to solve the problems that occur when identifying a product using a plurality of images generated by a plurality of imaging means.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Mode for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[0016] [Embodiment 1] FIG. 1 is a block diagram illustrating the product registration device 2000 according to Embodiment 1. In FIG. 1, each block represents a configuration in terms of function units, not hardware units.

[0017] The merchandise registration device 2000 includes a plurality of imaging units 2020 and a recognition unit 2030. The imaging unit 2020 captures an image of a merchandise to generate an image. Hereinafter, this image is referred to as a merchandise image.

[0018] The imaging range of each imaging unit 2020 includes part or all of the merchandise passing area. The merchandise passing area is a space through which an operator who operates the merchandise registration device 2000 passes the merchandise to be registered for settlement. Here, the operator may be a store clerk or a customer.

[0019] FIG. 2 is a diagram illustrating the merchandise passing area. The platform 20 is a platform for placing a merchandise basket or the like. The merchandise passing area 10 is the merchandise passing area. Thus, for example, the merchandise passing area is a space on the platform 20. However, the merchandise passing area 10 only needs to be a space through which an operator passes the merchandise to allow the imaging unit 2020 to read merchandise information, and is not limited to the space illustrated in FIG. 2. Further, the store that operates the merchandise registration device 2000 is not limited to a store that uses a merchandise basket.

[0020] The recognition unit 2030 recognizes a merchandise using the merchandise image generated by each imaging unit 2020. The recognition of a merchandise means identifying the merchandise and registering the merchandise as a settlement target.

[0021] For example, the recognition unit 2030 identifies a merchandise by analyzing an image of a merchandise information symbol shown in the merchandise image. The merchandise information symbol is a symbol for recognizing information related to a merchandise. Here, the symbol is a barcode, a two-dimensional code (such as a QR code (registered trademark)), or a character string symbol, etc. Note that the character string mentioned here includes a numerical sequence. And the merchandise information symbol is a barcode or the like in which information for recognizing merchandise information (such as an ID of merchandise information) is encoded, or a character string symbol representing information for recognizing merchandise information, etc.

[0022] For example, the recognition unit 2030 identifies a product by performing object recognition on an image of the product itself shown in the product image.

[0023] <Function and Effect> The product registration device 2000 of the present embodiment includes a plurality of imaging units 2020. Therefore, if a product is imaged by any one of the imaging units 2020, there is a possibility that the recognition unit 2030 can recognize the product. For example, when the operator moves the product in the product passage area, assume that the product is not arranged within the imaging range of a certain imaging unit 2020. In this case, if the product registration device 2000 does not have another imaging unit 2020, this product will not be recognized by the recognition unit 2030 and will not be registered as a settlement target. Therefore, the operator needs to redo the operation of causing the imaging unit 2020 to read the product information. On the other hand, if the product registration device 2000 has another imaging unit 2020, there is a possibility that the product will be arranged within the imaging range of the other imaging unit 2020, so there is a possibility that the recognition unit 2030 can recognize the product.

[0024] Also, as described above, the recognition of a product using a product image is performed by analyzing a product information symbol or identifying a product by object recognition. Here, in order to analyze a product information symbol, the product information symbol needs to be shown in the product image. Also, in order to identify a product by object recognition, a characteristic part of the product needs to be shown in the product image. For example, when identifying the type of canned coffee by object recognition, it is not possible to identify with a product image that only shows the bottom of the can, and a product image showing a label or the like is required. According to the product registration device 2000 of the present embodiment, if a product information symbol or a characteristic part of a product is imaged by any one of the imaging units 2020, the recognition unit 2030 can recognize the product.

[0025] Therefore, according to the merchandise registration device 2000 of the present embodiment, the probability that a product is recognized by the recognition unit 2030 is higher compared to the case where only one imaging unit 2020 is provided. Therefore, since the probability that a product is registered as a settlement target is increased, the efficiency of the operation of causing the merchandise registration device 2000 to recognize a product is increased.

[0026] Hereinafter, the merchandise registration device 2000 of the present embodiment will be described in more detail.

[0027] <Example of the hardware configuration of the merchandise registration device 2000> The merchandise registration device 2000 may be realized only by hardware (e.g., a hard-wired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program for controlling it, etc.).

[0028] The merchandise registration device 2000 is implemented using a dedicated terminal such as a cash register terminal, for example. However, the merchandise registration device 2000 may be implemented using various general-purpose computers such as a PC (Personal Computer) or a server machine instead of such a dedicated terminal.

[0029] FIG. 3 is a block diagram illustrating the hardware configuration of a computer 1000 used for implementing the product registration apparatus 2000. The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage 1080, and an input / output interface 1100. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage 1080, and the input / output interface 1100 to transmit and receive data to and from each other. However, the method of connecting the processor 1040 and the like to each other is not limited to bus connection. The processor 1040 is an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 1060 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 1080 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card. Also, the storage 1080 may be a memory such as a RAM or a ROM. The storage 1080 stores various data and programs.

[0030] The input / output interface 1100 is an interface for connecting the computer 1000 and an input / output device. The computer 1000 is connected to a keyboard, a mouse, a display, or the like via the input / output interface 1100.

[0031] <<Hardware Configuration of the Imaging Unit 2020>> For example, the imaging unit 2020 is realized by a camera 90 having an image sensor. Also, for example, the imaging unit 2020 may be realized by a barcode reader. In this case, the imaging unit 2020 irradiates light onto a product and receives the reflected light with a light receiving element to generate data representing the barcode pattern. The product image also includes data representing this barcode pattern.

[0032] <Arrangement of the Imaging Unit 2020> The number of imaging units 2020 is any number of two or more. Hereinafter, specific arrangements of the imaging units 2020 in the cases where the number of imaging units 2020 is two, four, and eight respectively will be exemplified. Note that the distance between the imaging units 2020 in each arrangement is arbitrary. For example, this distance is 5 cm.

[0033] <<Arrangement 1>> FIG. 4 is a first perspective view illustrating the arrangement of two imaging units 2020. In FIG. 4, the x direction is the direction in which the operator moves the product. The y direction is the depth direction of the stage 20. The z direction is the vertical direction.

[0034] FIG. 5 is a plan view corresponding to the arrangement of FIG. 4. FIG. 5(a) is a yz plan view of FIG. 4, and FIG. 5(b) is an xy plan view of FIG. 4. The imaging range 30 represents the imaging range of the imaging unit 2020. The imaging direction 40 is a direction starting from the imaging unit 2020 and passing through the center of the imaging range 30 of the imaging unit 2020. Hereinafter, the imaging direction 40 of the imaging unit 2020 will also be referred to as "the orientation of the imaging unit 2020".

[0035] As shown in FIG. 5(a), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are different positions. Specifically, the position of the imaging unit 2020-1 in the y direction is one end of the stage 20 in the y direction, and the position of the imaging unit 2020-2 in the y direction is the other end of the stage 20 in the y direction. On the other hand, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the z direction are the same position. These are all installed on the stage 20. Also, as shown in FIG. 5(b), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the x direction are the same position.

[0036] As shown in FIG. 5(a), the orientations of the imaging unit 2020-1 and the imaging unit 2020-2 are both directions of looking up at the product passing area 10 obliquely from below in the yz plane view. Note that, as shown in FIG. 5(b), it is preferable that a part of the imaging ranges 30 of the imaging unit 2020-1 and the imaging unit 2020-2 overlap each other.

[0037] With such an arrangement, the product is imaged from a plurality of different directions in the y direction. Therefore, compared with the case where the product is imaged only from one direction, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, so the probability that the recognition unit 2030 can recognize the product is increased.

[0038] Note that the orientation of each imaging unit 2020 in the arrangements of FIGS. 4 and 5 may be the z direction.

[0039] <<Arrangement 2>> FIG. 6 is a second perspective view illustrating the arrangement of the two imaging units 2020. Here, the holding unit 2040 is a member for holding the imaging unit 2020.

[0040] FIG. 7 is a plan view corresponding to the arrangement of FIG. 6. FIG. 7(a) is a yz plan view of FIG. 6, and FIG. 7(b) is an xy plan view of FIG. 6. As shown in FIG. 7(a), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are different positions, and the positional relationship is the same as that in the case of FIG. 4. Also, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the z direction are the same position, but are different positions from the case of FIG. 4. Specifically, the imaging unit 2020-1 and the imaging unit 2020-2 are located above the product passing area 10. As shown in FIG. 7(b), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the x direction are the same position.

[0041] As shown in FIG. 7(a), the orientations of both the imaging unit 2020-1 and the imaging unit 2020-2 are directions that look down on the product passing area 10 obliquely from above in the yz plane view. Note that, as shown in FIG. 7(b), it is preferable that the imaging ranges 30 of the imaging unit 2020-1 and the imaging unit 2020-2 overlap in a partial range.

[0042] With such an arrangement, the product is imaged from a plurality of different directions in the y direction. Therefore, compared with the case where the product is imaged only from one direction, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, so the probability that the recognition unit 2030 can recognize the product is increased.

[0043] Note that the orientation of each imaging unit 2020 in the arrangement of FIGS. 6 and 7 may be in the -z direction.

[0044] <<Arrangement 3>> FIG. 8 is a third perspective view illustrating the arrangement of two imaging units 2020. FIG. 9 is a plan view corresponding to the arrangement of FIG. 8. FIG. 9(a) is a yz plan view of FIG. 8, and FIG. 9(b) is an xy plan view of FIG. 8. As shown in FIG. 9(a), the y-direction positions of the imaging unit 2020-1 and the imaging unit 2020-2 are different positions, and the positional relationship is the same as in the case of FIG. 4. Also, the z-direction positions of the imaging unit 2020-1 and the imaging unit 2020-2 are different positions. Specifically, the imaging unit 2020-1 is installed on the base 20, and the imaging unit 2020-2 is held by the holding unit 2040 and is located above the product passage area 10. As shown in FIG. 9(b), the x-direction positions of the imaging unit 2020-1 and the imaging unit 2020-2 are different positions.

[0045] As shown in FIG. 9(a), the orientations of the imaging unit 2020-1 and the imaging unit 2020-2 face each other across the product passage area 10 in a yz plane view. Also, as shown in FIG. 9(b), the orientations of the imaging unit 2020-1 and the imaging unit 2020-2 face each other in an xy plane view. Note that it is preferable that the imaging ranges 30 of the imaging unit 2020-1 and the imaging unit 2020-2 overlap in part.

[0046] With such an arrangement, the product is imaged from a plurality of different directions in the y direction. Therefore, compared with the case where the product is imaged only from one direction, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, so the probability that the recognition unit 2030 can recognize the product is increased. Furthermore, since the product is imaged from both above and below, compared with the case where the product is imaged only from above or below, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, so the probability that the recognition unit 2030 can recognize the product is increased.

[0047] In the arrangements of FIGS. 8 and 9, the direction of the imaging unit 2020-1 may be set to the -z direction, and the direction of the imaging unit 2020-2 may be set to the z direction.

[0048] <<Arrangement 4>> FIG. 10 is a first perspective view illustrating the arrangement of the four imaging units 2020. FIG. 11 is a plan view corresponding to the arrangement of FIG. 10. FIG. 11(a) is a yz plan view of FIG. 10, and FIG. 11(b) is an xy plan view of FIG. 10. As shown in FIG. 11(a), in the y direction, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 are the same, and the positions of the imaging unit 2020-3 and the imaging unit 2020-4 are the same. Specifically, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are at one end of the base 20, and the positions of the imaging unit 2020-3 and the imaging unit 2020-4 in the y direction are at the other end of the base 20. In the z direction, the positions of the imaging unit 2020-1 and the imaging unit 2020-4 are the same, and the positions of the imaging unit 2020-2 and the imaging unit 2020-3 are the same. Specifically, the imaging unit 2020-1 and the imaging unit 2020-4 are held by the holding unit 2040 and are located above the product passage area 10, and the imaging unit 2020-2 and the imaging unit 2020-3 are installed on the base 20. As shown in FIG. 11(b), the positions of each imaging unit 2020 in the x direction are the same.

[0049] The imaging directions 40 of the imaging unit 2020-1 and the imaging unit 2020-3 face each other. Also, the imaging directions 40 of the imaging unit 2020-2 and the imaging unit 2020-4 face each other. It is preferable that the imaging directions 40 from the imaging unit 2020-1 to the imaging unit 2020-4 each face the center point of the imaging unit 2020-1 to the imaging unit 2020-4. Also, it is preferable that the imaging ranges 30 of each imaging unit 2020 overlap each other in part.

[0050] Since the four imaging units 2020 image the product from different directions respectively, compared with the case where the number of imaging units 2020 is two, the dead angles of the imaging units 2020 are reduced. Therefore, compared with the case where the number of imaging units 2020 is two, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, so the probability that the recognition unit 2030 can recognize the product is increased.

[0051] Note that the orientations of the respective imaging units 2020 in the arrangements shown in FIGS. 8 and 9 are not limited to the orientations shown in FIGS. 8 and 9. For example, the imaging unit 2020-1 and the imaging unit 2020-4 may face the z direction, and the imaging unit 2020-2 and the imaging unit 2020-3 may face the -z direction.

[0052] <<Arrangement 5>> FIG. 12 is a second perspective view illustrating the arrangement of the four imaging units 2020. FIG. 13 is a plan view corresponding to the arrangement of FIG. 12. FIG. 13(a) is a yz plan view of FIG. 12, and FIG. 13(b) is an xy plan view of FIG. 12. As shown in FIG. 13(a), these positional relationships in the y direction and the z direction are the same as those in the case of FIG. 11(a). On the other hand, as shown in FIG. 13(b), these positional relationships in the x direction are different from those in the case of FIG. 11(b). Specifically, the positional relationships of the imaging unit 2020-1 and the imaging unit 2020-4 in the x direction are different from those in the case of FIG. 11(b).

[0053] The relationship of the imaging directions 40 of the respective imaging units 2020 is the same as the relationship in FIGS. 8 and 9. Also, it is preferable that the imaging ranges 30 of the respective imaging units 2020 overlap with each other in part.

[0054] The arrangements of FIGS. 12 and 13 are different from the arrangements of FIGS. 10 and 11 in that the product is imaged from different directions in the x direction. Therefore, compared with the arrangements of FIGS. 10 and 11, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, so the probability that the recognition unit 2030 can recognize the product is increased.

[0055] Note that the orientations of the imaging units 2020 in the arrangements of FIGS. 12 and 13 are not limited to the orientations shown in FIGS. 12 and 13. For example, the imaging unit 2020-1 and the imaging unit 2020-2 may face corresponding orientations, and the imaging unit 2020-3 and the imaging unit 2020-4 may face opposite orientations.

[0056] <<Arrangement 6>> FIG. 14 is a diagram illustrating the arrangement of eight imaging units 2020. FIG. 15 is a plan view corresponding to the arrangement of FIG. 14. FIG. 15(a) is a yz plan view of FIG. 14, and FIG. 15(b) is an xy plan view of FIG. 14. As shown in these figures, the positional relationships of the imaging units 2020-1 to 2020-4 and the positional relationships of the imaging units 2020-5 to 2020-8 are the same as the positional relationships in the arrangements of FIGS. 10 and 11, respectively. However, as shown in FIG. 15(b), the imaging units 2020-1 to 2020-4 and the imaging units 2020-5 to 2020-8 have different positions in the x direction.

[0057] The imaging directions 40 of the imaging unit 2020-1 and the imaging unit 2020-7, the imaging directions 40 of the imaging unit 2020-2 and the imaging unit 2020-8, the imaging directions 40 of the imaging unit 2020-3 and the imaging unit 2020-5, and the imaging directions 40 of the imaging unit 2020-4 and the imaging unit 2020-6 face each other, respectively. Note that it is preferable that the imaging directions 40 of all the imaging units 2020 face the central positions of all the imaging units 2020. Also, it is preferable that the imaging ranges 30 of the imaging units 2020 overlap with each other in part.

[0058] According to the arrangements of FIGS. 14 and 15, compared with the cases where the number of imaging units 2020 is two or four, the blind spots of the imaging units 2020 are reduced. Therefore, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, and the probability that the recognition unit 2030 can recognize the product is increased.

[0059] <<Regarding the holding unit 2040>> The holding unit 2040 is preferably composed of a member that allows visible light to pass through. For example, this member is a rod, column, or plate made of transparent plastic, glass, or the like. By allowing the holding unit 2040 to pass visible light, it is possible to prevent the commodity from being blocked from light by the holding unit 2040. As a result, it is possible to prevent the accuracy of recognizing the commodity by the recognition unit 2030 from decreasing.

[0060] <Flow of processing> FIG. 16 is a flowchart illustrating the flow of processing executed by the commodity registration device 2000 according to Embodiment 1. The imaging unit 2020 generates a commodity image (S102). The recognition unit 2030 recognizes the commodity using the commodity image (S104).

[0061] <Details of processing performed by imaging unit 2020> The imaging unit 2020 generates a commodity image (S102). The imaging unit 2020 may capture a still image or a moving image. In the latter case, the operator image is each frame constituting the moving image.

[0062] The timing at which the imaging unit 2020 performs imaging is various. For example, the imaging unit 2020 performs imaging at the timing when the operator causes the commodity registration device 2000 to recognize the commodity and before and after that. For example, an infrared sensor or the like that detects a person is provided near the commodity registration device 2000. The commodity registration device 2000 can grasp that there is an operator near the commodity registration device 2000 by receiving a notification from this infrared sensor. Therefore, for example, the imaging unit 2020 performs imaging from when it is detected by this infrared sensor that there is an operator near the commodity registration device 2000 until the operator is no longer detected by this infrared sensor.

[0063] Also, for example, the imaging unit 2020 may perform repeated imaging periodically. The frequency at which the imaging unit 2020 performs repeated imaging is, for example, the same as the frame rate of a general moving image, which is 1 / 30 second.

[0064] <Details of processing performed by recognition unit 2030> The recognition unit 2030 recognizes the product using the product image generated by the imaging unit 2020 (S104). For example, the recognition unit 2030 identifies the product by detecting the product information symbol shown in the product image and analyzing the detected product information symbol for each product image generated by each imaging unit 2020. Also, for example, the recognition unit 2030 identifies the product by performing object recognition on the product itself shown in the product image generated by each imaging unit 2020. Then, when the product can be identified from any of the product images, the identified product is registered as the settlement target.

[0065] Here, the specific results obtained for each of the plurality of product images may indicate different products. For example, assume that the specific result using the product image generated by a certain imaging unit 2020 shows the result "the product is X", and the specific result using the product image generated by another imaging unit 2020 shows the result "the product is Y". In this case, for example, the recognition unit 2030 identifies the product using the most likely result. For example, the recognition unit 2030 adopts the result with the most product images having the same specific result. For example, assume that the result of analyzing each of a certain three product images shows the result "the product is X", and the result of analyzing each of another five product images shows the result "the product is Y". In this case, the recognition unit 2030 adopts the result "the product is Y" with the larger number of product images and registers product Y as the settlement target. However, if the recognition unit 2030 cannot uniquely identify the product in this way, it may issue a warning.

[0066] [Embodiment 2] The product registration device 2000 of Embodiment 2 is represented in FIG. 1 in the same manner as the product registration device 2000 of Embodiment 1. The product registration device 2000 of Embodiment 2 is the same as the product registration device 2000 of Embodiment 1 except that the arrangement of the imaging unit 2020 is different.

[0067] In the product registration device 2000 according to Embodiment 2, among the plurality of imaging units 2020, at least two or more product registration devices 2000 are arranged to image products in different directions from each other. Hereinafter, the arrangement of the imaging unit 2020 will be specifically described.

[0068] <Arrangement 1> FIG. 17 is a perspective view illustrating the arrangement of three imaging units 2020. The x-direction, y-direction, and z-direction are the direction in which the operator moves the product, the depth direction of the base 20, and the vertical direction, respectively. The imaging range 30 is the imaging range of the imaging unit 2020. The imaging direction 40 is the direction starting from the imaging unit 2020 and passing through the center of the imaging range 30 of the imaging unit 2020.

[0069] As shown in FIG. 17, the imaging directions 40 of the respective imaging units 2020 are different from each other. And each imaging unit 2020 is housed in the same housing 100.

[0070] FIG. 18 is a plan view corresponding to the arrangement of FIG. 17. FIG. 18(a) is a yz plan view of FIG. 17, and FIG. 18(b) is an xy plan view of FIG. 17. As shown in FIG. 18(b), each imaging unit 2020 images products 50 in different directions from each other.

[0071] By arranging the imaging units 2020 in this way, since the products are imaged from a plurality of different directions from each other, the probability that the product information symbol or the characteristic part of the product appears in the product image is increased, and the probability that the recognition unit 2030 can recognize the product is increased. Further, as shown in the arrangements of FIGS. 17 and 18, since the plurality of imaging units 2020 can be arranged in a close position, the plurality of imaging units 2020 can be housed in the same housing. Therefore, the installation of the imaging unit 2020 becomes easy.

[0072] <Arrangement 2> FIG. 19 is a perspective view illustrating the arrangement of the eight imaging units 2020. In FIG. 19, the imaging units 2020-1 to 2020-3 are housed in the same housing 100-1. Similarly, the imaging units 2020-4 to 2020-6 are housed in the same housing 100-2. The housing 100-1 and the housing 100-2 are installed so as to face each other. The imaging unit 2020-7 is installed on the base 20. On the other hand, the imaging unit 2020-8 is installed above the product passage area 10. Although not shown, the imaging unit 2020-8 is held by the holding unit 2040.

[0073] FIG. 20 is a plan view corresponding to the arrangement of FIG. 19. FIG. 20(a) is a yz plan view of FIG. 19, and FIG. 20(b) is an xy plan view of FIG. 19. As shown in FIG. 20(b), the imaging units 2020-1 to 2020-3 image products 50 in different directions from each other, similar to the arrangement of FIG. 17. Also, the imaging units 2020-4 to 2020-6 image products 50 in different directions from each other. The imaging unit 2020-2 and the imaging unit 2020-5, and the imaging unit 2020-3 and the imaging unit 2020-6 image the product 50 at the same position from different directions from each other. Also, the imaging unit 2020-1, the imaging unit 2020-4, the imaging unit 2020-7, and the imaging unit 2020-8 image the product 50 at the same position from different directions from each other.

[0074] According to the arrangements of FIGS. 19 and 20, since the product 50 can be imaged from various directions by more imaging units 2020 than in the cases of FIGS. 17 and 18, the probability that the product information symbol or the characteristic part of the product appears in the product image increases, and the probability that the recognition unit 2030 can recognize the product increases.

[0075] The hardware configuration of the product registration device 2000 according to Embodiment 2 is the same as that of the product registration device 2000 according to Embodiment 1, except for the arrangement of the imaging units 2020.

[0076] Also, the processes executed by the respective functional components of the product registration device 2000 according to Embodiment 2 are the same as the processes executed by the respective functional components of the product registration device 2000 according to Embodiment 1.

[0077] <Function and Effect> According to the product registration device 2000 of the present embodiment, since the product is imaged from different directions by a plurality of imaging units 2020, the probability that the product information symbol or the characteristic part of the product appears in the product image is higher than that in the case where the number of imaging units 2020 is one. Therefore, the probability that the recognition unit 2030 can recognize the product is increased. Further, in the product registration device 2000 of the present embodiment, a plurality of product registration devices 2000 are installed at the same or near positions, and these imaging units 2020 image products in different directions. Therefore, since these imaging units 2020 can be housed in a common housing, the installation of the imaging units 2020 becomes easy.

[0078] [Embodiment 3] FIG. 21 is a block diagram illustrating a product registration device 2000 according to Embodiment 3. In FIG. 21, each block represents a configuration of a functional unit, not a configuration of a hardware unit.

[0079] The product registration device 2000 of Embodiment 3 includes a control unit 2060. The control unit 2060 images the operation of the operator to generate an image. Hereinafter, the image generated by the control unit 2060 is referred to as an operator image. Then, the control unit 2060 controls the imaging unit 2020 using the operator image.

[0080] <Hardware Configuration of Product Registration Device 2000> The hardware configuration of the product registration device 2000 of Embodiment 3 is represented by FIG. 3 in the same manner as the hardware configuration of the product registration device 2000 of Embodiment 1.

[0081] The control unit 2060 has an image sensor for generating an operator image. For example, the control unit 2060 is implemented using a camera having an image sensor. FIG. 22 is a diagram specifically illustrating the product registration device 2000 of Embodiment 3. In the product registration device 2000 of FIG. 22, a camera 110 for realizing the control unit 2060 is installed on the holding unit 2040. However, the installation position of the camera 110 is not limited to the position shown in FIG. 22.

[0082] Note that the camera included in the control unit 2060 may be one or a plurality.

[0083] Here, the resolution of the imaging element of the control unit 2060 may be lower than the resolution of the imaging element of the imaging unit 2020. This is because, as will be described later, the resolution of the operator image that can be used for detecting the position of the product or the product information symbol only needs to be lower than the resolution of the product image that is required to identify the product, as long as it can be used for such detection. By lowering the resolution of the imaging element of the control unit 2060 in this way, the manufacturing cost of the product registration device 2000 can be reduced. However, the resolution of the imaging element of the control unit 2060 may also be equal to or higher than the resolution of the imaging element of the imaging unit 2020.

[0084] The storage 1080 of Embodiment 3 further stores program modules for realizing each functional component of Embodiment 3. Then, the processor 1040 realizes the functions of each functional component of Embodiment 4 by executing these program modules.

[0085] <Flow of processing> FIG. 23 is a flowchart illustrating the flow of processing executed by the product registration device 2000 of Embodiment 3. The control unit 2060 generates an operator image (S202). The control unit 2060 controls the imaging unit 2020 based on the operator image (S204). The imaging unit 2020 images a product based on the control by the control unit 2060 and generates a product image (S206). The recognition unit 2030 recognizes the product using the generated product image (S208).

[0086] <Details of processing performed by the control unit 2060> The control unit 2060 generates an operator image (S202). The control unit 2060 may capture a still image or a moving image. In the latter case, the operator image is each frame constituting the moving image.

[0087] The timing at which the control unit 2060 performs imaging varies. This timing is the same as the timing at which the imaging unit 2020 performs imaging as described in Embodiment 1.

[0088] Furthermore, the control unit 2060 controls the imaging unit 2020 using the operator image (S204). There are various specific methods by which the control unit 2060 controls the imaging unit 2020. Hereinafter, the methods will be exemplified and described.

[0089] <<Control Method 1>> The control unit 2060 controls the timing at which each imaging unit 2020 images the product. First, the control unit 2060 uses the operator image to detect the product and calculate the moving direction and moving speed of the product. Here, for the method of detecting the product from the image, technologies such as object recognition can be utilized. Also, the control unit 2060 calculates the moving direction and moving speed of the product based on the change in the position of the product shown in a plurality of operator images.

[0090] Then, the control unit 2060 uses the imaging time of the operator image, the moving direction of the product, and the moving speed of the product to calculate, for each imaging unit 2020, the timing at which the product is arranged within the imaging range of the imaging unit 2020. Then, the control unit 2060 causes each imaging unit 2020 to image the product at the timing calculated for each imaging unit 2020. By doing so, since each imaging unit 2020 can be operated at a timing when the probability that the recognition unit 2030 can recognize the product is high, the probability that the recognition unit 2030 can recognize the product increases.

[0091] Also, for the imaging unit 2020 in which the product is not arranged within the imaging range, the control unit 2060 may not cause the product to be imaged. By doing so, it is possible to suppress the power consumption of the imaging unit 2020 and prevent the deterioration of the imaging unit 2020.

[0092] Here, when the product is imaged by the imaging unit 2020, since the product is held in the operator's hand, a part of the surface of the product is covered by the operator's hand. Therefore, in the operator image, the product may appear in a state where a part of the product is missing.

[0093] In such a case, a method of performing control to determine the imaging timing of the imaging unit 2020 after complementing the part not shown in the operator image to reproduce the entire product is also conceivable. However, it is preferable that the control unit 2060 does not perform such complementation and determines the imaging timing of the imaging unit 2020 using the area of the product shown in the operator image. This is because the probability that the part covered by the operator's hand or the like does not appear in the product image generated by the imaging unit 2020 is high, and it is not appropriate to cause the imaging unit 2020 to image the product at the timing when such a part is arranged.

[0094] Therefore, for example, the control unit 2060 determines the imaging timing of each imaging unit 2020 on the assumption that only the area of the part of the product area shown in the operator image represents the product. By doing so, the control unit 2060 can cause the imaging unit 2020 to perform imaging at the timing when the area of the product shown in the product image (the part not covered by the hand or the like) is arranged within the imaging range. In addition, the control unit 2060 can cause only the imaging unit 2020 in which the area of the product shown in the product image is arranged within the imaging range to perform imaging.

[0095] Note that the moving speed of the product may be set in advance as a fixed value instead of being calculated using the operator image. For example, before starting the operation of the product registration device 2000, developers or the like of the product registration device 2000 perform an operation test to cause the product registration device 2000 to recognize the product. By doing so, the developers or the like repeatedly measure the moving speed of the product when the product registration device 2000 recognizes the product, and determine the above fixed value using the results. In this case, the control unit 2060 performs the same control as described above using the imaging time of the operator image, the moving direction of the product determined from the operator image, and the moving speed of the product set in advance as a fixed value.

[0096] Also, the moving direction of the product may be set in advance as a fixed value. For example, this fixed value is the long side direction of the base 60.

[0097] Each of the above fixed values may be set in advance in the control unit 2060, or may be stored in a storage unit accessible from the control unit 2060. In the latter case, the control unit 2060 reads out and uses each fixed value from this storage unit.

[0098] Also, if the positions and imaging directions of a camera or the like possessed by the control unit 2060, as well as the positions and recognition directions of the imaging unit 2020, are fixed, it is possible to determine in advance which imaging unit 2020 will perform imaging when a product is detected at that position in association with the position of the product in the operator image. Therefore, the control method determined in this way in advance is stored in a storage unit accessible from the control unit 2060. In this case, when the control unit 2060 detects a product from the operator image, it controls the imaging unit 2020 by referring to the information stored in this storage unit.

[0099] <<Control Method 2>> The control unit 2060 calculates the position, moving speed, and moving direction of the product information symbol, and controls the imaging unit 2020 based on these. For example, when the product information symbol is arranged within the imaging range of the imaging unit 2020, the control unit 2060 causes the imaging unit 2020 to perform imaging. Also, for example, the control unit 2060 controls the imaging unit 2020 so that imaging is performed only on the imaging unit 2020 where the product information symbol is arranged within the imaging range, and imaging is not performed on the imaging unit 2020 where the product information symbol is not arranged within the imaging range. Here, the method for calculating the position, moving speed, and moving direction of the product information symbol is the same as the method for calculating the position, moving speed, and moving direction of the product. Also, based on the position, moving speed, and moving direction of the product information symbol, the method for determining the timing to cause each imaging unit 2020 to perform imaging or causing only some of the imaging units 2020 to perform imaging is the same as the method for determining the timing to cause each imaging unit 2020 to perform imaging or causing only some of the imaging units 2020 to perform imaging based on the position, moving speed, and moving direction of the product.

[0100] Here, the product information symbol may not appear in the operator image. For example, when the control unit 2060 images the operator's actions from diagonally above, the product information symbol attached under the product may not appear. Therefore, when the control unit 2060 detects the product from the operator image but cannot detect the product information symbol, the control unit 2060 may control the imaging unit 2020 so that imaging is performed on the imaging unit 2020 arranged at a position that is a blind spot of the camera of the control unit 2060, and imaging is not performed on the other imaging units 2020. The imaging unit 2020 arranged at a position that is a blind spot of the camera of the control unit 2060 can be grasped in advance from the position and imaging direction of the control unit 2060 and the arrangement of each imaging unit 2020.

[0101] According to the method of controlling the imaging unit 2020 based on the position of the product information symbol or the like in this way, imaging can be performed on the imaging unit 2020 at a more appropriate timing than the method of controlling the imaging unit 2020 based on the position of the product or the like, or imaging can be performed only on the imaging unit 2020 that can read the product information symbol. In this case, the recognition unit 2030 recognizes the product by analyzing the product information symbol.

[0102] <Function and Effect> According to the present embodiment, the imaging unit 2020 is controlled based on the operator image. Therefore, as described above, the probability that the recognition unit 2030 can recognize the product can be increased, the power consumption of the imaging unit 2020 can be reduced, and the deterioration of the imaging unit 2020 can be prevented.

[0103] [Embodiment 4] The product registration device 2000 of Embodiment 4 is represented by FIG. 21 in the same manner as the product registration device 2000 of Embodiment 3. The product registration device 2000 of Embodiment 4 is the same as the product registration device 2000 of Embodiment 3 except for the points described below.

[0104] The control unit 2060 of Embodiment 4 determines one or more product images to be used for product recognition based on the operator image and the position and orientation of each imaging unit 2020. The recognition unit 2030 recognizes the product using the product image determined by the control unit 2060.

[0105] <Hardware Configuration> The hardware configuration of the product registration device 2000 of Embodiment 4 is represented by FIG. 3 in the same manner as the hardware configuration of the product registration device 2000 of Embodiment 1. The storage 1080 of Embodiment 4 further stores program modules for realizing each functional configuration unit of Embodiment 4. Then, the processor 1040 realizes the functions of each functional configuration unit of Embodiment 4 by executing these program modules.

[0106] <Processing Flow> FIG. 24 is a flowchart illustrating the flow of processing executed by the product registration apparatus 2000 according to Embodiment 4. Each imaging unit 2020 generates a product image (S302). The control unit 2060 generates an operator image (S304). The control unit 2060 determines a product image to be used for product recognition using the operator image (S306). The recognition unit 2030 recognizes the product using the determined product image (S308).

[0107] <Details of the processing performed by the control unit 2060> The control unit 2060 determines a product image using the operator image (S306). For this purpose, the control unit 2060 calculates the timing at which the product is placed within the imaging range of each imaging unit 2020 in the same manner as the control unit 2060 of Embodiment 3. However, in this embodiment, the usage method of the timing calculated in this way is different from that of Embodiment 3. This will be described in detail below.

[0108] <<Control method 1>> For example, the control unit 2060 determines the timing at which the product is placed within the imaging range for each imaging unit 2020 in the same manner as the control method 1 described in Embodiment 3. Then, the control unit 2060 specifies, for each imaging unit 2020, the product image captured at the timing when the product is placed within the imaging range among the product images generated by that imaging unit 2020. Then, the control unit 2060 determines to use the specified product images for product recognition.

[0109] Also, for example, the control unit 2060 determines the imaging unit 2020 in which the product is placed within the imaging range in the same manner as the control method 1 described in Embodiment 3. Then, the control unit 2060 determines to use the product image generated by the imaging unit 2020 in which the product is placed within the imaging range for product recognition.

[0110] <<Control method 2>> For example, the control unit 2060 determines the timing at which the product information symbol is placed within the imaging range by the same method as the control method 2 described in Embodiment 3. Then, for each imaging unit 2020, the control unit 2060 identifies the product image captured at the timing when the product information symbol is placed within the imaging range among the product images generated by the imaging unit 2020. Then, the control unit 2060 determines to use the identified product images for product recognition.

[0111] Also, for example, the control unit 2060 determines the imaging unit 2020 in which the product information symbol is placed within the imaging range by the same method as the control method 2 described in Embodiment 3. Then, the control unit 2060 determines to use the product image generated by the imaging unit 2020 in which the product information symbol is placed within the imaging range for product recognition.

[0112] [Function and Effect] According to this embodiment, for the same reason as in Embodiment 3, the probability that the product registration device 2000 can recognize a product is increased. Furthermore, according to this embodiment, the control unit 2060 does not need to control the imaging unit 2020. Therefore, the control of the imaging unit 2020 by the product registration device 2000 is simplified.

[0113] [Embodiment 5] The product registration device 2000 of Embodiment 5 is represented in FIG. 21, similar to the product registration device 2000 of Embodiment 4. The product registration device 2000 of Embodiment 5 is the same as the product registration device 2000 of Embodiment 4, except for the points described below.

[0114] The control unit 2060 of Embodiment 4 calculates the timing at which the product or the product information symbol is placed within the imaging range of each imaging unit 2020. However, since the timing calculated by the control unit 2060 is a predicted value, the actual timing at which the product or the product information symbol is placed within the imaging range of each imaging unit 2020 may deviate from this predicted value.

[0115] In addition, the control unit 2060 of Embodiment 4 predicts the imaging unit 2020 in which the product or product information symbol is arranged within the imaging range by calculating the moving direction of the product or the like. However, since the moving direction of the product may change, the product may not be arranged within the imaging range of the imaging unit 2020 predicted to have the "product arranged within the imaging range", or the product may be arranged within the imaging range of the imaging unit 2020 predicted to have the "product not arranged within the imaging range".

[0116] Therefore, the control unit 2060 of Embodiment 5 weights the product images captured by the imaging unit 2020. Then, the recognition unit 2030 recognizes the product using each weighted product image.

[0117] <Flow of processing> FIG. 25 is a flowchart illustrating the flow of processing executed by the product registration device 2000 of Embodiment 5. The processing performed in S302 and S304 is the same as in the case of FIG. 23. The control unit 2060 weights each product image (S402). The recognition unit 2030 recognizes the product using the weighted product images (S404).

[0118] <Details of processing performed by control unit 2060> The control unit 2060 of Embodiment 5 weights each product image using the operator image (S402). Specifically, each control method described in Embodiment 3 and Embodiment 4 is used to weight the product image. For example, the control unit 2060 makes the weight of the imaging unit 2020 predicted to have the "product or product information symbol arranged within the imaging range" larger than the weight of the imaging unit 2020 predicted to have the "product or product information symbol not arranged within the imaging range". In addition, the control unit 2060 increases the weight of the product image captured closer to the timing when it is predicted that the product or product information symbol is arranged within the imaging range.

[0119] Here, let the weight assigned to the product image i be ai based on whether the control unit 2060 determines that the product or the product information symbol is placed within the imaging range, and let the weight assigned to the product image i be bi based on the deviation between the timing when the control unit 2060 determines that the product or the product information symbol is placed within the imaging range and the imaging time of the product image. In this case, for example, the control unit 2060 assigns a weight of ai * bi to the product image i. However, the control unit 2060 may assign only ai or bi to the product image i.

[0120] <Details of the processing performed by the recognition unit 2030> The recognition unit 2030 recognizes the product using the weighted product images (S404). Here, the recognition unit 2030 uses the weight assigned to the product image as an index of the likelihood of the product recognized using that product image. Specifically, for each product image in which the same product is shown, the recognition unit 2030 identifies the product shown in that product image. Then, when the product cannot be uniquely identified, the recognition unit 2030 uniquely identifies the product by using the weight assigned to each product image as the specific likelihood based on that product image.

[0121] For example, among a plurality of product images generated for a certain product, assume that for some of the product images, it is identified that "the product is X" through object recognition or the like. On the other hand, assume that for another product image, it is identified that "the product is Y" based on the result of object recognition or the like. In this case, the recognition unit 2030 calculates the above two specific likelihoods using the weights assigned to each product image, and determines the more likely result as the product identification result. For example, the recognition unit 2030 uses the weight of the product image for which it is identified that "the product is X" and the weight of the product image for which it is identified that "the product is Y" as the specific likelihoods of each identification. Then, the recognition unit 2030 determines the identification with the larger weight as the final identification result.

[0122] In the above example, assume that there are a plurality of product images specified as "the product is X" and a plurality of product images specified as "the product is Y", respectively. In this case, the recognition unit 2030 compares the total weight value of the product images specified as "the product is X" with the total weight value of the product images specified as "the product is Y", and determines the specification with the larger total weight value as the final identification result.

[0123] <Function and Effect> According to the product registration device 2000 of the present embodiment, by weighting each product image using the predicted value of the timing when the product is placed within the imaging range of the imaging unit 2020, etc., the product can be uniquely identified with high accuracy. Therefore, the accuracy of product recognition by the product registration device 2000 is improved.

[0124] [Embodiment 6] FIG. 26 is a block diagram illustrating the settlement system 4000 according to Embodiment 6. In FIG. 1, each block represents a configuration in terms of functional units, not hardware units.

[0125] The settlement system 4000 includes any one of the product registration devices 2000 according to Embodiments 1 to 5 and a settlement device 3000. The settlement device 3000 is a device that performs settlement processing on the products recognized by the recognition unit 2030.

[0126] The product registration device 2000 generates settlement information used for settlement of the products registered as settlement targets by being recognized by the recognition unit 2030. Here, a plurality of products may be included in the settlement targets in one settlement process. For example, after receiving an operation instructing the start of the registration process of the settlement target, the product registration device 2000 registers one or more products recognized by the recognition unit 2030 until receiving an operation instructing the end of the registration process of the settlement target as the settlement targets in one settlement process. The settlement information regarding a certain settlement process indicates the IDs of each product registered as the target of the settlement process. The settlement information may further indicate a transaction number, the amount of each product, and the total amount.

[0127] The settlement device 3000 is used for the settlement process of the products registered as the settlement targets. Specifically, the settlement device 3000 performs processes such as presenting the total amount, accepting the input of the payment, counting the input payment, returning the change, and issuing a receipt.

[0128] The settlement device 3000 may be provided integrally with the product registration device 2000 or separately. When the settlement device 3000 is provided integrally with the product registration device 2000, the computer 1000 has not only the function of operating as the product registration device 2000 but also the function of operating as the settlement device 3000.

[0129] When the settlement device 3000 is provided separately from the product registration device 2000, the settlement device 3000 may be implemented by a dedicated terminal such as a cash register terminal or by a general-purpose computer. The hardware configuration of the computer that realizes the settlement device 3000 is the same as, for example, the hardware configuration of the computer 1000 in FIG. 3. Further, devices for a customer to input the payment, devices for returning the change, and devices for issuing a receipt are connected to the input / output interface of the computer that realizes the settlement device 3000.

[0130] As described above, embodiments of the present invention have been described with reference to the drawings, but these are examples of the present invention, and various configurations other than the above can also be adopted.

[0131] Examples of reference embodiments are appended below. 1. A product registration device having a plurality of imaging means for imaging a product to generate a product image at mutually different positions, and recognition means for recognizing the product using an image of the product information symbol, which is an image of the product shown in the product image generated by each of the imaging means or a symbol attached to the product for identifying the product. 2. The product registration device according to 1., wherein the first imaging means and the second imaging means image the product from mutually different directions. 3. The merchandise registration device according to 1. or 2., wherein the first imaging means and the second imaging means are provided at positions facing each other with a merchandise passage area through which the merchandise passes therebetween. 4. A plurality of imaging means for imaging a merchandise to generate a merchandise image, and recognition means for recognizing the merchandise using an image of a merchandise information symbol which is an image of the merchandise or a symbol attached to the merchandise and specifying the merchandise, shown in the merchandise image generated by each said imaging means. Each said imaging means images merchandise in different directions, the merchandise registration device. 5. The merchandise registration device according to any one of 1. to 4., wherein each said imaging means includes, in an imaging range, a part or all of a merchandise passage area through which the merchandise passes. 6. A holding means for holding the imaging means and allowing visible light to pass through, wherein the visible light passing through the holding means passes through a merchandise passage area through which the merchandise passes, the merchandise registration device according to any one of 1. to 5. 7. The merchandise registration device according to any one of 1. to 6., having control means for imaging an operator's action to generate an operator image and using the operator image to cause some of the plurality of said imaging means to image the merchandise. 8. Control means for imaging an operator's action to generate an operator image and using the operator image to determine a merchandise image to be used for merchandise recognition from among the merchandise images imaged by each imaging means, wherein the recognition means performs merchandise recognition using the merchandise image determined by the control means, the merchandise registration device according to any one of 1. to 6. 9. Having control means for imaging an operator's action to generate an operator image and using the operator image to determine a merchandise image to be used for merchandise recognition, wherein the recognition means performs merchandise recognition using each said weighted merchandise image, the merchandise registration device according to any one of 1. to 6. 10. A program for operating a computer as the merchandise registration device according to any one of 1. to 9. 11. A control method to be executed by a computer, An imaging step in which imaging means provided at different positions image a product to generate a product image; A recognition step of recognizing the product using an image of a product information symbol that is an image of the product or a symbol attached to the product and specifying the product, which is shown in the product image generated by each of the imaging means. A control method having the above. 12. The control method according to 11., wherein the first imaging means and the second imaging means image the product from different directions. 13. The control method according to 11. or 12., wherein the first imaging means and the second imaging means are provided at positions facing each other with a product passage area through which the product passes therebetween. 14. A control method executed by a computer, the method including: an imaging step in which a plurality of imaging means image a product to generate a product image; A recognition step of recognizing the product using an image of a product information symbol that is an image of the product or a symbol attached to the product and specifying the product, which is shown in the product image generated by each of the imaging means. The method has the above, Each of the imaging means images products in different directions. A control method. 15. The control method according to any one of claims 11. to 14., wherein each of the imaging means includes, in an imaging range, a part or all of a product passage area through which the product passes. 16. Having holding means for holding the imaging means and allowing visible light to pass through, The visible light that has passed through the holding means passes through a product passage area through which the product passes. The control method according to any one of 11. to 15. 17. The control method according to any one of 11. to 16., having a control step of imaging an operator's operation to generate an operator image and using the operator image to cause some of the plurality of imaging means to image a product. 18. An imaging step of imaging an operator's operation to generate an operator image, and a control step of using the operator image to determine a product image to be used for product recognition from among the product images imaged by each imaging means; The recognition step is the control method according to any one of 11. to 16., which performs recognition of a product using the product image determined by the control step. 19. A control step of capturing an operation of an operator to generate an operator image, and determining a product image to be used for product recognition using the operator image. The recognition step is the control method according to any one of 11. to 16., which performs recognition of a product using each of the weighted product images. 20. A settlement system including the product registration device according to any one of 1. to 9. and a settlement device that performs settlement processing of the product recognized by the recognition means of the product registration device.

Explanation of Signs

[0132] 10 Product passage area 20 units 30 Imaging range 40 Imaging direction 50 Product 60 units 90 Camera 100 Housing 110 Camera 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage 1100 Input / output interface 2000 Product registration device 2020 Imaging unit 2030 Recognition unit 2040 Holding unit 2060 Control unit

Claims

1. The camera has a plurality of image capturing means at different positions for capturing images of the products and generating images thereof; a recognition means for recognizing the commodity by using an image of the commodity or an image of a commodity information symbol which is attached to the commodity and is a symbol for identifying the commodity, which is captured in an image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; The information processing device further includes a control means for identifying the timing at which the product depicted in the operator image is placed within the imaging range of each of the imaging means based on changes in the position of the product depicted in multiple time-series operator images of the operator, and causing each of the imaging means to capture an image at the identified timing.

2. The camera has a plurality of image capturing means at different positions for capturing images of the products and generating images thereof; a recognition means for recognizing the commodity by using an image of the commodity or an image of a commodity information symbol which is attached to the commodity and is a symbol for identifying the commodity, which is captured in an image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; The information processing device further has a control means for identifying the timing at which the product depicted in the operator image is placed within the imaging range of each of the imaging means based on changes in the position of the product depicted in multiple time-series operator images of the operator, and determining that the image captured and generated at the identified timing is the image to be used for product recognition.

3. The plurality of imaging means include A first imaging means for capturing an image of a commodity to be checked out and generating an image; 3. The information processing apparatus according to claim 1, further comprising second and third imaging means provided at different positions in the vertical and horizontal directions from the first imaging means, for capturing images of the products to be checked out and generating images.

4. The information processing device according to claim 1 , wherein an imaging range of each of the plurality of imaging means includes a part or an entirety of a commodity passing area through which the commodity to be checked out passes in order to be registered.

5. The information processing device according to claim 3 , wherein the first imaging means, the second imaging means, and the third imaging means image the commodity to be settled from different directions.

6. The first imaging means images the product to be checked out from above, The information processing device according to claim 5 , wherein the second imaging means and the third imaging means image the commodity to be checked out from the side.

7. The information processing apparatus according to claim 6 , wherein the second imaging means and the third imaging means face each other across an area in which a commodity to be settled is recognized.

8. a computer used with a plurality of image capturing means provided at different positions for capturing images of products and generating images; Recognizing the product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, the image being captured in the image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; The control method includes identifying a timing at which a commodity captured in an operator image is placed within an imaging range of each of the imaging means based on a change in the position of the commodity captured in a plurality of time-series operator images of the operator, and causing each of the imaging means to capture an image at the identified timing.

9. a computer used with a plurality of image capturing means provided at different positions for capturing images of products and generating images; Recognizing the product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, the image being captured in the image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; A control method which identifies the timing at which the product depicted in the operator image is placed within the imaging range of each of the imaging means based on changes in the position of the product depicted in multiple time-series operator images of the operator, and determines the image captured and generated at the identified timing as the image to be used for product recognition.

Images