Information processor and control method

By employing multiple imaging units positioned differently to capture product images from various angles, the system addresses the challenge of incomplete recognition in single-device systems, enhancing accuracy and completeness of product identification.

JP2024159948AActive Publication Date: 2024-11-08NEC CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024148325
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-08
Estimated Expiration
2035-03-31

AI Technical Summary

Technical Problem

Existing product recognition systems using a single imaging device face challenges in accurately identifying products due to potential blind spots and incomplete capture of product information symbols or characteristics, leading to reduced recognition accuracy.

Method used

Implementing multiple imaging units positioned differently in vertical and horizontal directions to capture product images from various angles, ensuring comprehensive coverage and increased chances of capturing product information symbols or characteristics, with a recognition unit to notify discrepancies in recognition results.

Benefits of technology

Enhances product recognition accuracy by minimizing blind spots and improving the likelihood of capturing essential product details, thereby increasing the probability of successful identification and registration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024159948000001_ABST
    Figure 2024159948000001_ABST
Patent Text Reader

Abstract

To solve a problem that arises when using multiple images generated by multiple imaging means to identify commodities.SOLUTION: An information processor includes an acquisition unit, a recognition unit 2030, and a notification unit. The acquisition unit acquires an image of a commodity to be settled that is captured by a plurality of imaging units 2020. The recognition unit 2030 recognizes the commodities based on the images. The notification unit performs notification when recognition results based on the images captured by each of the imaging units 2020 are different.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing apparatus and a control method. [Background technology]

[0002] Cash register terminals operated by operators themselves are beginning to be used in stores such as supermarkets. The operator registers the products to be purchased by having the cash register terminal recognize the barcodes or other items attached to the products. The operator then purchases the registered products by inserting the amount displayed on the screen into the cash register terminal.

[0003] An example of a document disclosing technology related to such cash register terminals is Patent Document 1. Patent Document 1 discloses technology that has one scanner and one camera, and uses the results of scanning a product and the results of capturing an image of the product by the camera to determine whether the scanned product is the same as the captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-160493 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to improve the accuracy of recognizing products, it is conceivable to identify products using multiple images generated by multiple imaging means.

[0006] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to solve problems that arise when identifying a product using a plurality of images generated by a plurality of imaging means. [Means for solving the problem]

[0007] The first information processing device of the present invention comprises an acquisition means for acquiring images of the product to be checked out captured by multiple imaging means, a recognition means for recognizing the product based on the images, and a notification means for issuing a notification when the recognition results based on the images captured by each imaging means differ.

[0008] A second information processing device of the present invention comprises an acquisition means for acquiring images of a product to be checked out captured by a first imaging means, a second imaging means, and a third imaging means, and a recognition means for recognizing the product based on the images, wherein the second imaging means and the third imaging means are arranged at different positions vertically and horizontally from the first imaging means.

[0009] A third information processing device of the present invention comprises a first imaging means for imaging a commodity to be checked out, a second imaging means and a third imaging means arranged at different vertical and horizontal positions from the first imaging means for imaging the commodity, and an output means for outputting information relating to the commodity recognized based on images captured by the first imaging means, the second imaging means, and the third imaging means.

[0010] In a first control method of the present invention, a computer acquires images of a product to be checked out captured by multiple imaging means, recognizes the product based on the images, and notifies if the recognition results based on the images captured by each of the imaging means differ.

[0011] A second control method of the present invention includes a computer acquiring images of a product to be checked out captured by a first imaging means, a second imaging means, and a third imaging means, and recognizing the product based on the images, wherein the second imaging means and the third imaging means are positioned at different vertical and horizontal positions from the first imaging means.

[0012] A third control method of the present invention involves a computer capturing an image of a product to be checked out using a first imaging means, capturing an image of the product using a second imaging means and a third imaging means that are provided at different vertical and horizontal positions from the first imaging means, and outputting information regarding the product recognized based on images captured by the first imaging means, the second imaging means, and the third imaging means. Effect of the Invention

[0013] According to the present invention, it is possible to solve problems that arise when identifying a product using a plurality of images generated by a plurality of imaging means. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram illustrating a product registration device according to a first embodiment. [Diagram 2] FIG. 13 is a diagram illustrating an example of a product passing area. [Diagram 3] FIG. 2 is a block diagram illustrating a hardware configuration of a computer used to implement the product registration device. [Figure 4] FIG. 2 is a first perspective view illustrating an arrangement of two recognition units. [Diagram 5] FIG. 5 is a plan view corresponding to the arrangement of FIG. [Figure 6] FIG. 13 is a second perspective view illustrating the arrangement of two recognition units. [Figure 7] FIG. 7 is a plan view corresponding to the arrangement of FIG. 6. [Figure 8] FIG. 13 is a third perspective view illustrating the arrangement of two recognition units. [Figure 9] FIG. 9 is a plan view corresponding to the arrangement of FIG. [Figure 10] FIG. 2 is a first perspective view illustrating an arrangement of four recognition units. [Figure 11] FIG. 11 is a plan view corresponding to the arrangement of FIG. [Figure 12] FIG. 13 is a second perspective view illustrating an arrangement of four recognition units. [Figure 13] FIG. 13 is a plan view corresponding to the arrangement of FIG. 12. [Figure 14] FIG. 13 is a diagram illustrating an example of the arrangement of eight recognition units. [Figure 15] FIG. 15 is a plan view corresponding to the arrangement of FIG. [Figure 16] 4 is a flowchart illustrating a process executed by the product registration device of the first embodiment. [Figure 17] FIG. 13 is a perspective view illustrating an example of an arrangement of three recognition units. [Figure 18] FIG. 17 is a plan view corresponding to the arrangement of FIG. 16. [Figure 19] FIG. 13 is a perspective view illustrating an example of an arrangement of eight recognition units. [Figure 20] FIG. 19 is a plan view corresponding to the arrangement of FIG. 18. [Figure 21] FIG. 11 is a block diagram illustrating a product registration device according to a third embodiment. [Figure 22] FIG. 11 is a diagram specifically illustrating a product registration device according to a third embodiment. [Diagram 23] 13 is a flowchart illustrating a process executed by a product registration device of a third embodiment. [Figure 24] 13 is a flowchart illustrating a process executed by the product registration device of the fourth embodiment. [Diagram 25] 13 is a flowchart illustrating a process executed by the product registration device of the fifth embodiment. [Figure 26] FIG. 13 is a block diagram illustrating a settlement system according to a sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are given the same reference numerals and the description will be omitted as appropriate.

[0016] [Embodiment 1] Fig. 1 is a block diagram illustrating a product registration device 2000 according to embodiment 1. In Fig. 1, each block represents a functional configuration, not a hardware configuration.

[0017] The merchandise registration device 2000 has a plurality of imaging units 2020 and a recognition unit 2030. The imaging units 2020 capture images of merchandise to generate images. Hereinafter, these images are referred to as merchandise images.

[0018] The imaging range of each imaging unit 2020 includes a part or the whole of the commodity passing area. The commodity passing area is a space through which an operator of the commodity registration device 2000 passes a commodity to be settled in order to register the commodity. Here, the operator may be a store clerk or a customer.

[0019] FIG. 2 is a diagram illustrating a product passing area. The platform 20 is a platform on which a shopping basket or the like is placed. The product passing area 10 is a product passing area. In this manner, for example, the product passing area is the space above the platform 20. However, the product passing area 10 is not limited to the space illustrated in FIG. 2 as long as it is a space through which an operator passes a product to have the image capturing unit 2020 read the product information. Furthermore, stores that operate the product registration device 2000 are not limited to stores that use shopping baskets.

[0020] The recognition unit 2030 recognizes products using the product images generated by each imaging unit 2020. Recognizing a product means identifying the product and registering the product as a payment target.

[0021] For example, the recognition unit 2030 identifies a product by analyzing an image of a product information symbol shown in a product image. The product information symbol is a symbol that recognizes information about a product. Here, the symbol is a barcode, a two-dimensional code (such as QR Code (registered trademark)), or a character string symbol. Note that the character string here also includes a numeric string. The product information symbol is a barcode or the like in which information that recognizes the product information (such as the ID of the product information) is encoded, or a character string symbol that represents information that recognizes the product information.

[0022] Furthermore, for example, the recognition unit 2030 identifies a product by performing object recognition on an image of the product itself shown in the product image.

[0023] <Actions and Effects> The commodity registration device 2000 of this embodiment has a plurality of imaging units 2020. Therefore, if a commodity is imaged by any of the imaging units 2020, there is a possibility that the recognition unit 2030 can recognize the commodity. For example, when an operator moves a commodity in a commodity passing area, the commodity is not placed within the imaging range of a certain imaging unit 2020. In this case, if the commodity registration device 2000 does not have another imaging unit 2020, the commodity is not recognized by the recognition unit 2030 and is not registered as a settlement target. Therefore, the operator needs to redo the operation of making the imaging unit 2020 read the commodity information. On the other hand, if the commodity registration device 2000 has another imaging unit 2020, there is a possibility that the commodity is placed within the imaging range of the other imaging unit 2020, and therefore the recognition unit 2030 can recognize the commodity.

[0024] As described above, the recognition of a product using a product image is performed by analyzing a product information symbol or identifying a product by object recognition. Here, in order to analyze a product information symbol, the product information symbol must appear in the product image. In addition, in order to identify a product by object recognition, a characteristic part of the product must appear in the product image. For example, when identifying a type of canned coffee by object recognition, a product image showing only the bottom of the can is not sufficient to identify the type of canned coffee, and a product image showing a label or the like is required. According to the product registration device 2000 of this embodiment, if a product information symbol or a characteristic part of a product is imaged by any of the imaging units 2020, the recognition unit 2030 can recognize the product.

[0025] Therefore, according to the merchandise registration device 2000 of this embodiment, the probability that the merchandise will be recognized by the recognition unit 2030 is higher than when only one imaging unit 2020 is provided. Therefore, the probability that the merchandise will be registered as a payment target is higher, and the efficiency of the work of having the merchandise registration device 2000 recognize the merchandise is improved.

[0026] The product registration device 2000 of this embodiment will be described in further detail below.

[0027] <Example of Hardware Configuration of Product Registration Device 2000> The product registration device 2000 may be realized using only hardware (e.g., a hardwired electronic circuit, etc.) or a combination of hardware and software (e.g., a combination of an electronic circuit and a program to control it, etc.).

[0028] The product registration device 2000 is implemented using a dedicated terminal such as a cash register terminal. However, the product registration device 2000 may be implemented using various general-purpose computers such as a PC (Personal Computer) or a server machine instead of such a dedicated terminal.

[0029] FIG. 3 is a block diagram illustrating a hardware configuration of a computer 1000 used to implement the product registration device 2000. The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage 1080, and an input / output interface 1100. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage 1080, and the input / output interface 1100 to transmit and receive data to and from each other. However, the method of connecting the processor 1040 and the like to each other is not limited to bus connection. The processor 1040 is an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 1060 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 1080 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card. The storage 1080 may also be a memory such as a RAM or a ROM. The storage 1080 stores various data and programs.

[0030] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. The computer 1000 is connected to a keyboard, a mouse, a display, or the like via the input / output interface 1100.

[0031] <<Hardware configuration of the imaging unit 2020>> For example, the imaging unit 2020 is realized by a camera 90 having an imaging element. Alternatively, for example, the imaging unit 2020 may be realized by a barcode reader. In this case, the imaging unit 2020 irradiates light onto a product and receives the reflected light with a light receiving element to generate data representing a barcode pattern. The product image also includes the data representing this barcode pattern.

[0032] <Arrangement of the imaging unit 2020> The number of the imaging units 2020 is any number equal to or greater than two. Specific examples of the arrangement of the imaging units 2020 are given below for cases where the number of imaging units 2020 is two, four, and eight. Note that the distance between the imaging units 2020 in each arrangement is arbitrary. For example, this distance is 5 cm.

[0033] <<Placement 1>> Fig. 4 is a first perspective view illustrating an example of the arrangement of two imaging units 2020. In Fig. 4, the x direction is the direction in which the operator moves the product, the y direction is the depth direction of the stage 20, and the z direction is the vertical direction.

[0034] Fig. 5 is a plan view corresponding to the arrangement in Fig. 4. Fig. 5(a) is a yz plan view of Fig. 4, and Fig. 5(b) is an xy plan view of Fig. 4. An imaging range 30 represents the imaging range of the imaging unit 2020. An imaging direction 40 is a direction starting from the imaging unit 2020 and passing through the center of the imaging range 30 of the imaging unit 2020. Hereinafter, the imaging direction 40 of the imaging unit 2020 will also be referred to as the "orientation of the imaging unit 2020".

[0035] As shown in FIG. 5(a), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are different. Specifically, the position of the imaging unit 2020-1 in the y direction is one end of the stage 20 in the y direction, and the position of the imaging unit 2020-2 in the y direction is another end of the stage 20 in the y direction. On the other hand, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the z direction are the same. Both of these are installed on the stage 20. Also, as shown in FIG. 5(b), the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the x direction are the same.

[0036] As shown in Fig. 5(a), the orientations of the imaging units 2020-1 and 2020-2 are both in a direction looking up from diagonally below the commodity passage area 10 in the yz plane view. Note that, as shown in Fig. 5(b), it is preferable that the imaging ranges 30 of the imaging units 2020-1 and 2020-2 partially overlap each other.

[0037] With this arrangement, the product is imaged from a plurality of different directions in the y direction, which increases the probability that the product information symbol or a characteristic part of the product will appear in the product image compared to when the product is imaged from only one direction, thereby increasing the probability that the recognition unit 2030 can recognize the product.

[0038] In addition, the direction of each imaging unit 2020 in the arrangement of FIG. 4 and FIG. 5 may be the z direction.

[0039] <<Placement 2>> 6 is a second perspective view illustrating an example of the arrangement of the two imaging units 2020. Here, the holding unit 2040 is a member for holding the imaging units 2020.

[0040] FIG. 7 is a plan view corresponding to the arrangement of FIG. 6. FIG. 7(a) is a yz plan view of FIG. 6, and FIG. 7(b) is an xy plan view of FIG. 6. As FIG. 7(a) shows, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the y direction are different positions, and the positional relationship is the same as in FIG. 4. Also, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the z direction are the same position, but are different positions from the case of FIG. 4. Specifically, the imaging unit 2020-1 and the imaging unit 2020-2 are located above the commodity passage area 10. As FIG. 7(b) shows, the positions of the imaging unit 2020-1 and the imaging unit 2020-2 in the x direction are the same position.

[0041] As shown in Fig. 7(a), the orientations of the imaging units 2020-1 and 2020-2 are both in a direction looking diagonally down from above the commodity passage area 10 in the yz plane view. Note that, as shown in Fig. 7(b), it is preferable that the imaging ranges 30 of the imaging units 2020-1 and 2020-2 overlap in part.

[0042] With this arrangement, the product is imaged from a plurality of different directions in the y direction, which increases the probability that the product information symbol or a characteristic part of the product will appear in the product image compared to when the product is imaged from only one direction, thereby increasing the probability that the recognition unit 2030 can recognize the product.

[0043] It should be noted that the orientation of each imaging unit 2020 in the arrangements of FIGS. 6 and 7 may be in the -z direction.

[0044] <<Placement 3>> FIG. 8 is a third perspective view illustrating the arrangement of the two imaging units 2020. FIG. 9 is a plan view corresponding to the arrangement of FIG. 8. FIG. 9(a) is a yz plan view of FIG. 8, and FIG. 9(b) is an xy plan view of FIG. 8. As shown in FIG. 9(a), the positions of the imaging units 2020-1 and 2020-2 in the y direction are different positions, and the positional relationship is the same as in FIG. 4. In addition, the positions of the imaging units 2020-1 and 2020-2 in the z direction are different positions. Specifically, the imaging unit 2020-1 is installed on the platform 20, and the imaging unit 2020-2 is held by the holding unit 2040 and is located above the commodity passage area 10. As shown in FIG. 9(b), the positions of the imaging units 2020-1 and 2020-2 in the x direction are different positions.

[0045] As shown in Fig. 9(a), the imaging units 2020-1 and 2020-2 face each other across the commodity passage area 10 in the yz plane. As shown in Fig. 9(b), the imaging units 2020-1 and 2020-2 also face each other in the xy plane. Note that it is preferable that the imaging ranges 30 of the imaging units 2020-1 and 2020-2 overlap in part.

[0046] With this arrangement, the product is imaged from a plurality of different directions in the y direction. Therefore, compared to when the product is imaged from only one direction, the probability that the product information symbol or a characteristic part of the product will appear in the product image is higher, and the recognition unit 2030 is more likely to recognize the product. Furthermore, since the product is imaged from both above and below, compared to when the product is imaged only from above or below, the probability that the product information symbol or a characteristic part of the product will appear in the product image is higher, and the recognition unit 2030 is more likely to recognize the product.

[0047] In the arrangements of FIGS. 8 and 9, the imaging unit 2020-1 may be oriented in the -z direction, and the imaging unit 2020-2 may be oriented in the z direction.

[0048] <<Placement 4>> FIG. 10 is a first perspective view illustrating an arrangement of four imaging units 2020. FIG. 11 is a plan view corresponding to the arrangement of FIG. 10. FIG. 11(a) is a yz plan view of FIG. 10, and FIG. 11(b) is an xy plan view of FIG. 10. As shown in FIG. 11(a), in the y direction, the positions of the imaging units 2020-1 and 2020-2 are the same, and the positions of the imaging units 2020-3 and 2020-4 are the same. Specifically, the positions of the imaging units 2020-1 and 2020-2 in the y direction are one end of the stage 20, and the positions of the imaging units 2020-3 and 2020-4 in the y direction are another end of the stage 20. In the z direction, the positions of the imaging units 2020-1 and 2020-4 are the same, and the positions of the imaging units 2020-2 and 2020-3 are the same. Specifically, the imaging units 2020-1 and 2020-4 are held by a holder 2040 and positioned above the commodity passage area 10, and the imaging units 2020-2 and 2020-3 are installed on a platform 20. As shown in Fig. 11(b) , the positions of the imaging units 2020 in the x direction are the same.

[0049] The imaging directions 40 of the imaging units 2020-1 and 2020-3 face each other. The imaging directions 40 of the imaging units 2020-2 and 2020-4 face each other. Note that it is preferable that the imaging directions 40 of the imaging units 2020-1 to 2020-4 face the center points of the imaging units 2020-1 to 2020-4, respectively. It is also preferable that the imaging ranges 30 of the imaging units 2020 overlap each other in part.

[0050] In this way, the four imaging units 2020 capture images of the product from different directions, which reduces blind spots of the imaging units 2020 compared to when there are two imaging units 2020. Therefore, compared to when there are two imaging units 2020, there is a higher probability that the product information symbol or a characteristic part of the product will appear in the product image, which increases the probability that the recognition unit 2030 can recognize the product.

[0051] Note that the orientation of each imaging unit 2020 in the arrangement shown in Fig. 8 and Fig. 9 is not limited to the orientation shown in Fig. 8 and Fig. 9. For example, the imaging unit 2020-1 and the imaging unit 2020-4 may face the z direction, and the imaging unit 2020-2 and the imaging unit 2020-3 may face the -z direction.

[0052] <<Placement 5>> FIG. 12 is a second perspective view illustrating an arrangement of four imaging units 2020. FIG. 13 is a plan view corresponding to the arrangement of FIG. 12. FIG. 13(a) is a yz plan view of FIG. 12, and FIG. 13(b) is an xy plan view of FIG. 12. As shown in FIG. 13(a), the positional relationship in the y direction and the z direction is the same as in FIG. 11(a). On the other hand, as shown in FIG. 13(b), the positional relationship in the x direction is different from that in FIG. 11(b). Specifically, the positions in the x direction of the imaging unit 2020-1 and the imaging unit 2020-4 are different from that in FIG. 11(b).

[0053] The relationship between the imaging directions 40 of the imaging sections 2020 is similar to that in FIGS. It is also preferable that the imaging ranges 30 of the imaging units 2020 partially overlap each other.

[0054] 12 and 13 differ from the arrangement in Figures 10 and 11 in that the product is imaged from a different direction in the x direction. Therefore, compared to the arrangement in Figures 10 and 11, the probability that the product information symbol or a characteristic part of the product will appear in the product image is higher, and the recognition unit 2030 is more likely to recognize the product.

[0055] Note that the orientation of each imaging unit 2020 in the arrangement of Fig. 12 and Fig. 13 is not limited to the orientation shown in Fig. 12 and Fig. 13. For example, the imaging unit 2020-1 and the imaging unit 2020-2 may face in corresponding directions, and the imaging unit 2020-3 and the imaging unit 2020-4 may face in opposing directions.

[0056] <<Placement 6>> FIG. 14 is a diagram illustrating an example of the arrangement of eight imaging units 2020. FIG. 15 is a plan view corresponding to the arrangement of FIG. 14. FIG. 15(a) is a yz plan view of FIG. 14, and FIG. 15(b) is an xy plan view of FIG. 14. As shown in these figures, the positional relationship between the imaging unit 2020-1 to the imaging unit 2020-4 and the positional relationship between the imaging unit 2020-5 to the imaging unit 2020-8 are the same as the positional relationship in the arrangement of FIG. 10 and FIG. 11. However, as shown in FIG. 15(b), the positions of the imaging unit 2020-1 to the imaging unit 2020-4 and the imaging unit 2020-5 to the imaging unit 2020-8 in the x direction are different.

[0057] The imaging directions 40 of the imaging units 2020-1 and 2020-7, the imaging directions 40 of the imaging units 2020-2 and 2020-8, the imaging directions 40 of the imaging units 2020-3 and 2020-5, and the imaging directions 40 of the imaging units 2020-4 and 2020-6 are opposed to each other. Note that it is preferable that the imaging directions 40 of each imaging unit 2020 are all directed toward the center positions of all the imaging units 2020. It is also preferable that the imaging ranges 30 of each imaging unit 2020 overlap each other in part.

[0058] 14 and 15, the blind spots of the imaging unit 2020 are reduced compared to when the number of imaging units 2020 is two or four. Therefore, the probability that the product information symbol or a characteristic part of the product will appear in the product image increases, and therefore the recognition unit 2030 is more likely to recognize the product.

[0059] <<Regarding the holding unit 2040>> The holding unit 2040 is preferably made of a material that transmits visible light. For example, this material is a rod, a pillar, or a plate made of transparent plastic, glass, or the like. By making the holding unit 2040 transmit visible light, it is possible to prevent the holding unit 2040 from blocking light from reaching the product. As a result, it is possible to prevent a decrease in the accuracy of product recognition by the recognition unit 2030.

[0060] <Processing flow> 16 is a flowchart illustrating a process executed by the product registration device 2000 of the embodiment 1. The imaging unit 2020 generates a product image (S102). The recognition unit 2030 recognizes a product using the product image (S104).

[0061] <Details of the processing performed by the imaging unit 2020> The imaging unit 2020 generates a product image (S102). The imaging unit 2020 may capture a still image or a moving image. In the latter case, the operator image is each frame constituting the moving image.

[0062] The timing at which the imaging unit 2020 captures an image varies. For example, the imaging unit 2020 captures an image at the timing when the operator makes the merchandise registration device 2000 recognize the merchandise, and before and after that. For example, an infrared sensor that detects a person is provided near the merchandise registration device 2000. By receiving a notification from this infrared sensor, the merchandise registration device 2000 can know that an operator is near the merchandise registration device 2000. Therefore, for example, the imaging unit 2020 captures an image from the time when the infrared sensor detects that the operator is near the merchandise registration device 2000 until the infrared sensor no longer detects the operator.

[0063] Also, for example, the imaging unit 2020 may periodically and repeatedly capture images. The frequency at which the imaging unit 2020 repeatedly captures images is, for example, 1 / 30 seconds, which is the same as the frame rate of a typical video.

[0064] <Details of the process performed by the recognition unit 2030> The recognition unit 2030 recognizes the product using the product image generated by the imaging unit 2020 (S104). For example, the recognition unit 2030 identifies the product by detecting a product information symbol appearing in each product image generated by each imaging unit 2020 and analyzing the detected product information symbol. In addition, for example, the recognition unit 2030 identifies the product by performing object recognition on the product itself appearing in the product image generated by each imaging unit 2020. Then, when the product can be identified from any of the product images, the identified product is registered as a payment target.

[0065] Here, the identification results performed for each of the multiple product images may indicate different products. For example, suppose that the identification result using a product image generated by a certain imaging unit 2020 indicates the result "the product is X", and the identification result using a product image generated by another imaging unit 2020 indicates the result "the product is Y". In this case, for example, the recognition unit 2030 identifies the product using the most likely result. For example, the recognition unit 2030 adopts the result with the most product images that have the same identification result. For example, suppose that the result of analyzing each of three product images indicates the result "the product is X", and the result of analyzing each of five other product images indicates the result "the product is Y". In this case, the recognition unit 2030 adopts the result "the product is Y" with the greater number of product images, and registers the product Y as the payment target. However, the recognition unit 2030 may issue a warning if the product cannot be uniquely identified in this way.

[0066] [Embodiment 2] The merchandise registration device 2000 of the second embodiment is shown in Fig. 1 in the same manner as the merchandise registration device 2000 of the first embodiment. The merchandise registration device 2000 of the second embodiment is similar to the merchandise registration device 2000 of the first embodiment, except that the arrangement of the imaging unit 2020 is different.

[0067] In the merchandise registration device 2000 of the second embodiment, at least two of the multiple image capturing units 2020 are arranged to capture images of merchandise in different directions. The arrangement of the image capturing units 2020 will be specifically described below.

[0068] <Placement 1> 17 is a perspective view illustrating an arrangement of three imaging units 2020. The x direction, y direction, and z direction are the direction in which the operator moves the product, the depth direction of the platform 20, and the vertical direction, respectively. The imaging range 30 is the imaging range of the imaging unit 2020. The imaging direction 40 is a direction that starts from the imaging unit 2020 and passes through the center of the imaging range 30 of the imaging unit 2020.

[0069] 17, the imaging directions 40 of the imaging units 2020 are different from one another. The imaging units 2020 are housed in the same housing 100.

[0070] Fig. 18 is a plan view corresponding to the arrangement in Fig. 17. Fig. 18(a) is a yz plan view of Fig. 17, and Fig. 18(b) is an xy plan view of Fig. 17. As shown in Fig. 18(b), each imaging unit 2020 captures an image of a commodity 50 in a different direction from each other.

[0071] By arranging the imaging units 2020 in this manner, the product is imaged from a plurality of different directions, increasing the probability that the product information symbol or a characteristic part of the product will appear in the product image, and increasing the probability that the recognition unit 2030 will be able to recognize the product. In addition, as shown in the arrangements of Figures 17 and 18, the multiple imaging units 2020 can be arranged in close proximity to each other, and therefore the multiple imaging units 2020 can be housed in the same housing. This makes it easier to install the imaging units 2020.

[0072] <Placement 2> FIG. 19 is a perspective view illustrating an example of the arrangement of eight imaging units 2020. In FIG. 19, imaging units 2020-1 to 2020-3 are housed in the same housing 100-1. Similarly, imaging units 2020-4 to 2020-6 are housed in the same housing 100-2. The housings 100-1 and 100-2 are installed so as to face each other. The imaging unit 2020-7 is installed on the platform 20. Meanwhile, the imaging unit 2020-8 is installed above the product passage area 10. Although not shown in the figure, the imaging unit 2020-8 is held by the holding unit 2040.

[0073] FIG. 20 is a plan view corresponding to the arrangement of FIG. 19. FIG. 20(a) is a yz plan view of FIG. 19, and FIG. 20(b) is an xy plan view of FIG. 19. As shown in FIG. 20(b), the imaging units 2020-1 to 2020-3 capture images of the commodity 50 in different directions, similar to the arrangement of FIG. 17. The imaging units 2020-4 to 2020-6 also capture images of the commodity 50 in different directions. The imaging units 2020-2 and 2020-5, and the imaging units 2020-3 and 2020-6 each capture images of the commodity 50 in the same position from different directions. The imaging units 2020-1, 2020-4, 2020-7, and 2020-8 capture images of the commodity 50 in the same position from different directions.

[0074] According to the arrangement of Figures 19 and 20, it is possible to image the product 50 from various directions using more imaging units 2020 than in the case of Figures 17 and 18, which increases the probability that the product information symbol or a characteristic part of the product will appear in the product image and increases the probability that the recognition unit 2030 will be able to recognize the product.

[0075] The hardware configuration of the merchandise registration device 2000 of the second embodiment is similar to that of the merchandise registration device 2000 of the first embodiment, except for the arrangement of the imaging unit 2020.

[0076] Moreover, the processes executed by the functional components of the product registration device 2000 of the second embodiment are similar to the processes executed by the functional components of the product registration device 2000 of the first embodiment.

[0077] <Actions and Effects> According to the commodity registration device 2000 of this embodiment, since the multiple imaging units 2020 capture images of commodities from different directions, the probability that the commodity information symbol or a characteristic part of the commodity will appear in the commodity image is higher than when there is only one imaging unit 2020. Therefore, the probability that the recognition unit 2030 can recognize the commodity is higher. Furthermore, in the commodity registration device 2000 of this embodiment, the multiple commodity registration devices 2000 are installed in the same or nearby positions, and these imaging units 2020 capture images of commodities in different directions. Therefore, these imaging units 2020 can be housed in a common housing, making it easy to install the imaging units 2020.

[0078] [Embodiment 3] Fig. 21 is a block diagram illustrating a product registration device 2000 according to embodiment 3. In Fig. 21, each block represents a functional configuration, not a hardware configuration.

[0079] The commodity registration device 2000 of the third embodiment includes a control unit 2060. The control unit 2060 captures an image of the operator's actions and generates an image. Hereinafter, the image generated by the control unit 2060 is referred to as an operator image. The control unit 2060 controls the imaging unit 2020 using the operator image.

[0080] <Hardware configuration of the product registration device 2000> The hardware configuration of the product registration device 2000 of the third embodiment is shown in FIG. 3, similar to the hardware configuration of the product registration device 2000 of the first embodiment.

[0081] The control unit 2060 has an imaging element for generating an image of the operator. For example, the control unit 2060 is implemented using a camera having an imaging element. FIG. 22 is a diagram specifically illustrating a product registration device 2000 of the third embodiment. In the product registration device 2000 of FIG. 22, a camera 110 for implementing the control unit 2060 is installed on the holding unit 2040. However, the installation position of the camera 110 is not limited to the position shown in FIG. 22.

[0082] The control unit 2060 may have one camera or multiple cameras.

[0083] Here, the resolution of the imaging element of the control unit 2060 may be lower than the resolution of the imaging element of the imaging unit 2020. This is because the resolution of the operator image, which may be used to detect the position of the product or product information symbol, etc., as described below, is lower than the resolution of the product image, which must be used to identify the product. By lowering the resolution of the imaging element of the control unit 2060 in this manner, the manufacturing cost of the product registration device 2000 can be reduced. However, the resolution of the imaging element of the control unit 2060 may be equal to or higher than the resolution of the imaging element of the imaging unit 2020.

[0084] The storage 1080 of the third embodiment further stores a program module for implementing each functional component of the third embodiment. The processor 1040 then executes each of these program modules to implement the functions of each functional component of the fourth embodiment.

[0085] <Processing flow> 23 is a flowchart illustrating a process executed by the product registration device 2000 of the third embodiment. The control unit 2060 generates an operator image (S202). The control unit 2060 controls the imaging unit 2020 based on the operator image (S204). The imaging unit 2020 captures an image of a product under the control of the control unit 2060, and generates a product image (S206). The recognition unit 2030 recognizes the product using the generated product image (S208).

[0086] <Details of the process performed by the control unit 2060> The control unit 2060 generates an operator image (S202). The control unit 2060 may capture a still image or a moving image. In the latter case, the operator image is each frame constituting the moving image.

[0087] There are various timings for the control unit 2060 to capture an image. The timings are similar to the timings for the image capture unit 2020 described in the first embodiment.

[0088] Furthermore, the control unit 2060 uses the operator image to control the imaging unit 2020 (S204). There are various specific methods by which the control unit 2060 controls the imaging unit 2020. Below, examples of such methods will be described.

[0089] <<Control method 1>> The control unit 2060 controls the timing at which each imaging unit 2020 captures an image of a commodity. First, the control unit 2060 detects the commodity and calculates the moving direction and moving speed of the commodity using the operator image. Here, a technique such as object recognition can be used as a method for detecting the commodity from the image. The control unit 2060 also calculates the moving direction and moving speed of the commodity based on the change in the position of the commodity captured in the multiple operator images.

[0090] Then, the control unit 2060 calculates, for each imaging unit 2020, the timing at which the product will be placed within the imaging range of the imaging unit 2020, using the imaging time of the operator image, the moving direction of the product, and the moving speed of the product. Then, the control unit 2060 causes each imaging unit 2020 to image the product at the timing calculated for each imaging unit 2020. In this way, each imaging unit 2020 can be operated at a timing at which the recognition unit 2030 is highly likely to recognize the product, thereby increasing the probability that the recognition unit 2030 will recognize the product.

[0091] Furthermore, the control unit 2060 may not cause the imaging unit 2020, which does not have a commodity placed within its imaging range, to capture an image of the commodity. This can reduce the power consumption of the imaging unit 2020 and prevent the imaging unit 2020 from deteriorating.

[0092] Here, when the image of the product is captured by the imaging unit 2020, the product is held in the operator's hand, and therefore a part of the surface of the product is covered by the operator's hand. Therefore, the image of the operator may show the product with a part of the product missing.

[0093] In such a case, a method can be considered in which the part not shown in the operator image is complemented to reproduce the entire product, and then control is performed to determine the imaging timing of the imaging unit 2020. However, it is preferable that the control unit 2060 does not perform such complementation, but instead determines the imaging timing of the imaging unit 2020 using the area of ​​the product shown in the operator image. This is because there is a high probability that the part covered by the operator's hand will not be shown in the product image generated by the imaging unit 2020, and therefore it is not appropriate to have the imaging unit 2020 capture the product at the timing when such a part is located.

[0094] Therefore, for example, the control unit 2060 determines the imaging timing of each imaging unit 2020 by regarding only the area of ​​the product that is shown in the operator image as representing the product. In this way, the control unit 2060 can cause the imaging units 2020 to capture images at a timing when the area of ​​the product shown in the product image (the part not covered by a hand, etc.) is located within the imaging range. Furthermore, the control unit 2060 can cause only the imaging units 2020 whose area of ​​the product shown in the product image is located within the imaging range to capture images.

[0095] The moving speed of the commodity may be set as a fixed value in advance, instead of being calculated using the operator image. For example, before starting operation of the commodity registration device 2000, the developer of the commodity registration device 2000 performs an operation test to make the commodity registration device 2000 recognize the commodity. In this way, the developer repeatedly measures the moving speed of the commodity when making the commodity registration device 2000 recognize the commodity, and determines the fixed value using the result. In this case, the control unit 2060 performs the same control as described above, using the imaging time of the operator image, the moving direction of the commodity calculated from the operator image, and the moving speed of the commodity set as a fixed value in advance.

[0096] The direction of movement of the product may also be set as a fixed value in advance. For example, this fixed value may be the long side direction of the platform 60.

[0097] The above-mentioned fixed values ​​may be preset in the control unit 2060, or may be stored in a storage unit accessible from the control unit 2060. In the latter case, the control unit 2060 reads out and uses the fixed values ​​from the storage unit.

[0098] Furthermore, if the positions and imaging directions of the cameras etc. owned by the control unit 2060 and the position and recognition direction of the imaging unit 2020 are fixed, it is possible to determine in advance which imaging unit 2020 should capture an image when a commodity is detected at a position in association with the position of the commodity in the operator image. Thus, such a predetermined control method is stored in a storage unit accessible by the control unit 2060. In this case, when the control unit 2060 detects a commodity from the operator image, it controls the imaging unit 2020 by referring to the information stored in the storage unit.

[0099] <<Control method 2>> The control unit 2060 calculates the position, movement speed, and movement direction of the commodity information symbol, and controls the imaging unit 2020 based on these. For example, the control unit 2060 causes the imaging unit 2020 to capture an image at a timing when the commodity information symbol is arranged within the imaging range of the imaging unit 2020. Also, for example, the control unit 2060 controls the imaging unit 2020 so that only the imaging unit 2020 in which the commodity information symbol is arranged within the imaging range captures an image, and does not cause the imaging unit 2020 in which the commodity information symbol is not arranged within the imaging range to capture an image. Here, the method of calculating the position, movement speed, and movement direction of the commodity information symbol is the same as the method of calculating the position, movement speed, and movement direction of the commodity information symbol. In addition, the method of determining the timing for each imaging unit 2020 to capture an image or of causing only some of the imaging units 2020 to capture an image based on the position, movement speed, and movement direction of the product information symbol is similar to the method of determining the timing for each imaging unit 2020 to capture an image or of causing only some of the imaging units 2020 to capture an image based on the position, movement speed, and movement direction of the product.

[0100] Here, the commodity information symbol may not appear in the operator image. For example, when the control unit 2060 captures an image of the operator's movement from diagonally above, the commodity information symbol attached under the commodity may not appear. Therefore, when the control unit 2060 detects a commodity from the operator image but cannot detect a commodity information symbol, the control unit 2060 may control the imaging unit 2020 so that the imaging unit 2020 arranged in a position in the blind spot of the camera of the control unit 2060 captures an image and the other imaging units 2020 do not capture an image. The imaging units 2020 arranged in a position in the blind spot of the camera of the control unit 2060 can be known in advance from the position and imaging direction of the control unit 2060 and the arrangement of each imaging unit 2020.

[0101] According to the method of controlling the imaging unit 2020 based on the position of the commodity information symbol in this way, it is possible to cause the imaging unit 2020 to capture an image at a more appropriate timing than the method of controlling the imaging unit 2020 based on the position of the commodity, etc., and to cause only the imaging unit 2020 that can read the commodity information symbol to capture an image. In this case, the recognition unit 2030 recognizes the commodity by analyzing the commodity information symbol.

[0102] <Actions and Effects> According to this embodiment, the imaging unit 2020 is controlled based on the operator image. Therefore, as described above, it is possible to increase the probability that the recognition unit 2030 can recognize the product, reduce the power consumption of the imaging unit 2020, and prevent the imaging unit 2020 from deteriorating.

[0103] [Embodiment 4] The merchandise registration device 2000 of the fourth embodiment is represented by Fig. 21 in the same manner as the merchandise registration device 2000 of the third embodiment. Except for the points described below, the merchandise registration device 2000 of the fourth embodiment is similar to the merchandise registration device 2000 of the third embodiment.

[0104] The control unit 2060 of the fourth embodiment determines one or more product images to be used for product recognition, based on the operator image and the positions and orientations of the imaging units 2020. The recognition unit 2030 uses the product images determined by the control unit 2060 to recognize the product.

[0105] <Hardware configuration> The hardware configuration of the product registration device 2000 of the fourth embodiment is shown in Fig. 3, similar to the hardware configuration of the product registration device 2000 of the first embodiment. The storage 1080 of the fourth embodiment further stores program modules for implementing each functional component of the fourth embodiment. The processor 1040 then executes each of these program modules to implement the functions of each functional component of the fourth embodiment.

[0106] <Processing flow> 24 is a flowchart illustrating a process executed by the product registration device 2000 of the fourth embodiment. Each imaging unit 2020 generates a product image (S302). The control unit 2060 generates an operator image (S304). The control unit 2060 uses the operator image to determine a product image to be used for product recognition (S306). The recognition unit 2030 recognizes the product using the determined product image (S308).

[0107] <Details of the process performed by the control unit 2060> The control unit 2060 determines the product image using the operator image (S306). To this end, the control unit 2060 calculates the timing at which the product is placed within the imaging range of each imaging unit 2020, etc., in the same manner as the control unit 2060 in the third embodiment. However, in this embodiment, the method of using the calculated timing, etc., differs from that in the third embodiment. This will be described in detail below.

[0108] <<Control method 1>> For example, the control unit 2060 determines the timing at which a commodity is placed within the imaging range for each imaging unit 2020 by a method similar to the control method 1 described in the third embodiment. Then, for each imaging unit 2020, the control unit 2060 identifies a commodity image that was captured at the timing at which the commodity was placed within the imaging range from among the commodity images generated by the imaging unit 2020. Then, the control unit 2060 decides to use each identified commodity image for commodity recognition.

[0109] Furthermore, for example, the control unit 2060 identifies the imaging unit 2020 within whose imaging range the commodity is placed, using a method similar to the control method 1 described in the embodiment 3. Then, the control unit 2060 determines to use the commodity image generated by the imaging unit 2020 within whose imaging range the commodity is placed, for commodity recognition.

[0110] <<Control method 2>> For example, the control unit 2060 determines the timing at which the commodity information symbol is arranged within the imaging range by a method similar to the control method 2 described in the embodiment 3. Then, for each imaging unit 2020, the control unit 2060 specifies a commodity image that was captured at the timing at which the commodity information symbol is arranged within the imaging range, from among the commodity images generated by the imaging unit 2020. Then, the control unit 2060 determines to use each specified commodity image for commodity recognition.

[0111] Furthermore, for example, the control unit 2060 identifies the imaging unit 2020 whose imaging range includes a commodity information symbol by a method similar to the control method 2 described in the embodiment 3. Then, the control unit 2060 determines to use the commodity image generated by the imaging unit 2020 whose imaging range includes a commodity information symbol for commodity recognition.

[0112] <Actions and Effects> According to this embodiment, the probability that the merchandise registration device 2000 can recognize the merchandise increases for the same reasons as in embodiment 3. Furthermore, according to this embodiment, the control unit 2060 does not need to control the imaging unit 2020. Therefore, the control of the imaging unit 2020 by the merchandise registration device 2000 becomes simple.

[0113] [Embodiment 5] The merchandise registration device 2000 of the fifth embodiment is shown in Fig. 21, similar to the merchandise registration device 2000 of the fourth embodiment. Except for the points described below, the merchandise registration device 2000 of the fifth embodiment is similar to the merchandise registration device 2000 of the fourth embodiment.

[0114] The control unit 2060 of the fourth embodiment calculates the timing at which the product or product information symbol is arranged within the imaging range of each imaging unit 2020. However, since the timing calculated by the control unit 2060 is a predicted value, the timing at which the product or product information symbol is actually arranged within the imaging range of each imaging unit 2020 may differ from this predicted value.

[0115] Furthermore, the control unit 2060 of the fourth embodiment predicts the imaging unit 2020 within whose imaging range the product or product information symbol will be placed by calculating the moving direction of the product, etc. However, since the moving direction of the product may change, the product may not be placed within the imaging range of the imaging unit 2020 predicted to "place the product within the imaging range," or the product may be placed within the imaging range of the imaging unit 2020 predicted to "not place the product within the imaging range."

[0116] Therefore, the control unit 2060 of the fifth embodiment weights the product images captured by the imaging unit 2020. Then, the recognition unit 2030 recognizes the product using each weighted product image.

[0117] <Processing flow> Fig. 25 is a flowchart illustrating a process executed by the product registration device 2000 of the fifth embodiment. The processes in S302 and S304 are the same as those in Fig. 23. The control unit 2060 assigns weights to each product image (S402). The recognition unit 2030 recognizes a product using the weighted product image (S404).

[0118] <Details of the process performed by the control unit 2060> The control unit 2060 of the fifth embodiment assigns weighting to each product image using the operator image (S402). Specifically, weighting is performed on the product images using each control method described in the third and fourth embodiments. For example, the control unit 2060 assigns a larger weight to the imaging unit 2020 predicted to have a "product or product information symbol placed within the imaging range" than to the imaging unit 2020 predicted to have a "product or product information symbol not placed within the imaging range". In addition, the control unit 2060 assigns a larger weight to a product image captured at a time closer to the time when the product or product information symbol is predicted to be placed within the imaging range.

[0119] Here, the weight that the control unit 2060 assigns to the product image i based on whether or not the product or product information symbol is placed within the imaging range is ai, and the weight that the control unit 2060 assigns to the product image i based on the discrepancy between the timing when the product or product information symbol is placed within the imaging range and the imaging time of the product image is bi. In this case, for example, the control unit 2060 assigns a weight of ai*bi to the product image i. However, the control unit 2060 may assign only ai or bi to the product image i.

[0120] <Details of the process performed by the recognition unit 2030> The recognition unit 2030 recognizes the product using the weighted product image (S404). Here, the recognition unit 2030 uses the weight assigned to the product image as an index of the likelihood of the product being recognized using that product image. Specifically, the recognition unit 2030 identifies the product shown in each product image that shows the same product. If the recognition unit 2030 is unable to uniquely identify the product, it uses the weight assigned to each product image as the likelihood of identification based on that product image, thereby uniquely identifying the product.

[0121] For example, among multiple product images generated for a certain product, some of the product images are identified as "product X" by object recognition or the like. Meanwhile, another product image is identified as "product Y" by the result of object recognition or the like. In this case, the recognition unit 2030 calculates the above two identification probabilities using the weights assigned to each product image, and the more probable result is the product identification result. For example, the recognition unit 2030 uses the weights of the product images identified as "product X" and the weights of the product images identified as "product Y" as the respective identification probabilities. Then, the recognition unit 2030 determines the identification with the larger weight as the final identification result.

[0122] In the above example, assume that there are multiple product images identified as "product X" and multiple product images identified as "product Y." In this case, the recognition unit 2030 compares the total weight value of the product images identified as "product X" and the total weight value of the product images identified as "product Y," and determines the identification with the larger total weight value as the final identification result.

[0123] <Actions and Effects> According to the commodity registration device 2000 of the present embodiment, it is possible to uniquely identify a commodity with high accuracy by weighting each commodity image using a predicted value of the timing when the commodity is placed within the imaging range of the imaging unit 2020. Therefore, the accuracy of commodity recognition by the commodity registration device 2000 is improved.

[0124] [Embodiment 6] Fig. 26 is a block diagram illustrating an adjustment system 4000 according to embodiment 6. In Fig. 1, each block represents a functional configuration, not a hardware configuration.

[0125] The settlement system 4000 includes any one of the merchandise registration devices 2000 according to the first to fifth embodiments, and a settlement device 3000. The settlement device 3000 is a device that performs settlement processing for the merchandise recognized by the recognition unit 2030.

[0126] The product registration device 2000 generates settlement information to be used for settlement of a product that has been registered as a settlement target by recognition by the recognition unit 2030. Here, the settlement target in one settlement process may include multiple products. For example, after receiving an operation to instruct the start of the registration process of the settlement target, the product registration device 2000 registers one or multiple products recognized by the recognition unit 2030 during the period from when the product registration device 2000 receives an operation to instruct the end of the registration process of the settlement target as a settlement target in one settlement process. The settlement information for a certain settlement process indicates the ID of each product registered as a target of the settlement process, etc. The settlement information may further indicate the transaction number, the amount of each product, the total amount, etc.

[0127] The settlement device 3000 is used for settlement processing of products registered as settlement targets. Specifically, the settlement device 3000 performs processing such as presenting the total amount, accepting the insertion of the payment, counting up the inserted payment, returning the change, and issuing a receipt.

[0128] The settlement device 3000 may be provided integrally with or separately from the merchandise registration device 2000. When the settlement device 3000 is provided integrally with the merchandise registration device 2000, the computer 1000 has not only a function to operate as the merchandise registration device 2000 but also a function to operate as the settlement device 3000.

[0129] When the settlement device 3000 is provided separately from the product registration device 2000, the settlement device 3000 may be implemented as a dedicated terminal such as a cash register terminal, or as a general-purpose computer. The hardware configuration of the computer that realizes the settlement device 3000 is, for example, similar to the hardware configuration of the computer 1000 in Fig. 3. Furthermore, a device for a customer to insert money, a device for refunding change, a device for issuing receipts, etc. are connected to the input / output interface of the computer that realizes the settlement device 3000.

[0130] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0131] Below, examples of reference forms are given. 1. The camera has a plurality of imaging means for capturing images of products and generating product images at different positions; The product registration device has a recognition means for recognizing a product using an image of the product or an image of a product information symbol, which is a symbol attached to the product and identifies the product, that appears in the product images generated by each of the imaging means. 2. A product registration device as described in 1., wherein the first imaging means and the second imaging means image the product from different directions. 3. A product registration device as described in 1. or 2., wherein the first imaging means and the second imaging means are provided in opposing positions across a product passing area through which products pass. 4. A plurality of imaging means for capturing images of products and generating product images; A recognition means for recognizing a product using an image of the product or an image of a product information symbol that is attached to the product and is a symbol for identifying the product, the image being captured in the product image generated by each of the imaging means; A product registration device, wherein each of the imaging means images a product in a different direction. 5. A product registration device described in any one of 1. to 4., wherein each of the imaging means has an imaging range that includes part or all of the product passing area through which the product passes. 6. A holding means for holding the imaging means and for transmitting visible light therethrough; The product registration device described in any one of 1. to 5., wherein the visible light that has passed through the holding means passes through a product passing area through which the product passes. 7. A product registration device described in any one of 1. to 6., having a control means for capturing an image of an operator by capturing an image of the operator, and using the image of the operator to cause some of the multiple imaging means to capture an image of a product. 8. A control means for capturing an image of an operator to generate an operator image, and using the operator image to determine a product image to be used for product recognition from among the product images captured by each imaging means; The product registration device according to any one of 1. to 6., wherein the recognition means recognizes the product using a product image determined by the control means. 9. A control means for capturing an image of an operator to generate an operator image and determining a commodity image to be used for commodity recognition using the operator image; The product registration device according to any one of 1. to 6., wherein the recognition means recognizes products using the weighted product images. 10. A program for causing a computer to operate as a product registration device described in any one of 1. to 9. 11. A control method implemented by a computer, comprising: An imaging step in which imaging means provided at different positions capture images of products to generate product images; A control method comprising: a recognition step of recognizing the product using an image of the product or an image of a product information symbol, which is a symbol attached to the product and identifies the product, that appears in the product image generated by each of the imaging means. 12. The control method described in claim 11, wherein the first imaging means and the second imaging means image the product from different directions. 13. A control method described in 11. or 12., wherein the first imaging means and the second imaging means are provided in opposing positions across a product passing area through which products pass. 14. A control method executed by a computer, comprising: an imaging step in which a plurality of imaging means capture images of products to generate product images; A recognition step of recognizing the product using an image of the product or an image of a product information symbol that is attached to the product and is a symbol that identifies the product, the image being captured in the product image generated by each of the imaging means, A control method in which each of the imaging means images a commodity in a different direction. 15. A control method according to any one of claims 11 to 14, wherein each of the imaging means has an imaging range that includes part or all of a product passing area through which the product passes. 16. A holding means for holding the imaging means and for transmitting visible light therethrough; A control method described in any one of 11. to 15., in which the visible light that has passed through the holding means passes through a product passing area through which the product passes. 17. A control method described in any one of claims 11 to 16, comprising a control step of capturing an image of an operator by capturing an image of the operator, and using the image of the operator to cause some of the multiple imaging means to capture an image of the commodity. 18. A control step of capturing an image of an operator to generate an operator image, and using the operator image to determine a product image to be used for product recognition from among the product images captured by each imaging means; The control method according to any one of claims 11 to 16, wherein the recognition step recognizes the product using a product image determined by the control step. 19. A control step of capturing an image of an operator to generate an operator image and using the operator image to determine a commodity image to be used for commodity recognition, The control method according to any one of claims 11 to 16, wherein the recognition step recognizes products using the weighted product images. 20. A settlement system having a product registration device described in any one of 1 to 9, and a settlement device that performs settlement processing for products recognized by the recognition means of the product registration device. [Explanation of symbols]

[0132] 10 Product Passing Area 20 units 30 Image Range 40 Imaging Direction 50 items 60 units 90 Camera 100 Case 110 Camera 1000 calculator 1020 Bus 1040 Processor 1060 Memory 1080 Storage 1100 Input / Output Interface 2000 Product registration device 2020 Imaging section 2030 Recognition part 2040 Holding part 2060 Control Unit

Claims

1. The camera has a plurality of image capturing means at different positions for capturing images of the products and generating images thereof; a recognition means for recognizing the commodity by using an image of the commodity or an image of a commodity information symbol which is attached to the commodity and is a symbol for identifying the commodity, which is captured in an image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; The information processing device further includes a control means for identifying the timing at which the product depicted in the operator image is placed within the imaging range of each of the imaging means based on changes in the position of the product depicted in multiple time-series operator images of the operator, and causing each of the imaging means to capture an image at the identified timing.

2. The camera has a plurality of image capturing means at different positions for capturing images of the products and generating images thereof; a recognition means for recognizing the commodity by using an image of the commodity or an image of a commodity information symbol which is attached to the commodity and is a symbol for identifying the commodity, which is captured in an image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; The information processing device further has a control means for identifying the timing at which the product depicted in the operator image is placed within the imaging range of each of the imaging means based on changes in the position of the product depicted in multiple time-series operator images of the operator, and determining that the image captured and generated at the identified timing is the image to be used for product recognition.

3. The plurality of imaging means include A first imaging means for capturing an image of a commodity to be checked out and generating an image; 3. The information processing apparatus according to claim 1, further comprising second and third imaging means provided at different positions in the vertical and horizontal directions from the first imaging means, for capturing images of the products to be checked out and generating images.

4. The information processing device according to claim 1 , wherein an imaging range of each of the plurality of imaging means includes a part or an entirety of a commodity passing area through which the commodity to be checked out passes in order to be registered.

5. The information processing device according to claim 3 , wherein the first imaging means, the second imaging means, and the third imaging means image the commodity to be settled from different directions.

6. The first imaging means images the product to be checked out from above, The information processing device according to claim 5 , wherein the second imaging means and the third imaging means image the commodity to be checked out from the side.

7. The information processing apparatus according to claim 6 , wherein the second imaging means and the third imaging means face each other across an area in which a commodity to be settled is recognized.

8. a computer used with a plurality of image capturing means provided at different positions for capturing images of products and generating images; Recognizing the product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, the image being captured in the image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; The control method includes identifying a timing at which a commodity captured in an operator image is placed within an imaging range of each of the imaging means based on a change in the position of the commodity captured in a plurality of time-series operator images of the operator, and causing each of the imaging means to capture an image at the identified timing.

9. a computer used with a plurality of image capturing means provided at different positions for capturing images of products and generating images; Recognizing the product using an image of the product or an image of a product information symbol that is attached to the product and identifies the product, the image being captured in the image generated by each of the imaging means; When the images generated by the imaging means are recognized to show different products, a notification is given; A control method which identifies the timing at which the product depicted in the operator image is placed within the imaging range of each of the imaging means based on changes in the position of the product depicted in multiple time-series operator images of the operator, and determines the image captured and generated at the identified timing as the image to be used for product recognition.

Citation Information

Patent Citations

  • Symbol recognition device and symbol recognition method

    JP2004199141A

  • Information processor and program

    JP2013054673A

  • Commodity data processing apparatus, commodity data processing method, and control program

    JP2013089084A

  • Method and mechanism for identifying objects

    JP2014531636A

  • Motion video recognition device and motion video recognition method

    WO2014049729A1