Item registration device and cart POS system equipped with the item registration device

Abstract

To provide an article registration device and a cart POS system mounting the article registration device which detect a feeding area of a commodity, perform photographing according to brightness of the area, and register an article within a photographed image.SOLUTION: An article registration device comprises an imaging device, a lighting unit, a position detection unit, and an information terminal. The imaging device photographs an article brought into a photographing area. The lighting unit illuminates the photographing area during a light-on period due to a set exposure amount. The position detection unit divides the photographing area into a plurality of areas, and detects the area where the article is placed in the presence of lighting of the lighting unit. The information terminal sets exposure time in the presence of the lighting of the lighting unit for each imaging device to photograph the article, so as to have a set exposure amount, based on brightness of lighting according to distance between the detected area and the imaging device, and recognizes and registers the article within a photographed image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to an article registration device that recognizes and registers articles in an image without exposure difference, and a cart POS system equipped with the article registration device.

Background Art

[0002] Normally, when purchasing a product, product registration is performed at a register installed in a store, and accounting is carried out. A cart POS (Point Of Sales) system has been proposed for retail stores equipped with shopping carts. Such a cart POS system is provided with an information terminal equipped with a touch panel, a barcode scanner, etc. on the shopping cart. When a user who is a customer using the shopping cart places an item to be purchased on the shopping cart, the identification code (barcode or two-dimensional barcode (QR code (registered trademark))) of the product to be purchased is read by the barcode scanner, and product registration is performed using barcode recognition technology.

[0003] Furthermore, within the scope of this system, there is a cart POS system that utilizes AI (Artificial Intelligence), which is called a frictionless cart, an AI cart, a smart cart, or an intelligent cart, etc. This shopping cart has a plurality of imaging devices arranged around the basket. When the product to be purchased is taken in and out of the basket, each device takes a picture of the product, recognizes the product package from the captured images, automatically identifies the type of the product, and uses package recognition technology to automatically register the product.

[0004] However, when multiple imaging devices photograph the same product at the same time, the brightness of the images captured by each device may differ depending on where the product is placed in the basket, for example, whether it is in the center or a corner of the basket. In other words, imaging devices closer to the product in the basket will capture brighter images, while imaging devices further away will capture darker images compared to those taken at close range. Furthermore, automatically adjusting the exposure time for each imaging device to prevent overexposure and blown-out highlights takes time, which could cause delays before product recognition can begin. There is also a possibility that the system may not be able to automatically adjust to the optimal brightness, potentially resulting in the product not being recognized. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Patent No. 6537054 [Overview of the project] [Problems that the invention aims to solve]

[0006] The problem that the embodiments of the present invention aim to solve is to provide an item registration device and a cart POS system equipped with the item registration device, which, when photographing items to be purchased with multiple cameras, adjusts the exposure timing of each camera according to the position in which the item is placed, thereby acquiring images without exposure differences, and registering items by package recognition or identification code recognition using the images. [Means for solving the problem]

[0007] An article registration device according to one embodiment photographs an article placed in a shooting area. multiple The imaging device includes, at least one illumination unit that illuminates the shooting area for a set exposure duration, a position detection unit that divides the shooting area into a plurality of areas and detects the area where the item is placed under the illumination of the illumination unit, and the detected area and the imaging device eachThe system includes an information terminal that, based on the brightness of the lighting corresponding to the distance from the imaging device, sets the exposure time under the illumination of the lighting unit to achieve the set exposure amount, takes a photograph, recognizes the object in the photographed image, and registers it. The information terminal sets the illumination period based on the set exposure amount, using the darkest area among the detected areas as a reference, and delays the exposure start time of the imaging device from the illumination start time of the illumination unit so that the exposure time becomes the set exposure amount for each imaging device. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an overall configuration diagram of a cart POS system equipped with an item registration device according to the first embodiment. [Figure 2] Figure 2 is a schematic diagram showing the main data structure of the information terminal table. [Figure 3] Figure 3 is a block diagram showing an example of the circuit configuration of an information terminal. [Figure 4] Figure 4 shows the overall configuration of a shopping cart in the first embodiment, in which cameras and lighting units are placed at the four corners of the basket. [Figure 5A] Figure 5A shows an example where cameras and lighting units are placed at the four corners of the basket, illustrating the brightness of the area as seen by the first camera. [Figure 5B] Figure 5B shows an example where cameras and lighting units are placed at the four corners of the basket, illustrating the brightness of the area as seen by the second camera. [Figure 5C] Figure 5C shows an example where cameras and lighting units are placed at the four corners of the basket, illustrating the brightness of the area as measured by the third camera. [Figure 5D] Figure 5D shows an example where cameras and lighting units are placed at the four corners of the basket, illustrating the brightness of the area as measured by the fourth camera. [Figure 6] Figure 6 shows the exposure timing of each camera when a product is placed in area A, in an example where cameras and lighting units are placed at the four corners of the basket. [Figure 7] Figure 7 shows the overall configuration of a shopping cart in the second embodiment, in which a camera and lighting unit are positioned in the center of the four sides of the basket opening. [Figure 8A]Figure 8A shows an example where the camera and lighting unit are placed in the center of the side of the basket, illustrating the brightness of the area as seen by the first camera. [Figure 8B] Figure 8B shows an example where the camera and lighting unit are placed in the center of the side of the basket, illustrating the brightness of the area as seen by the second camera. [Figure 8C] Figure 8C shows an example where the camera and lighting unit are placed in the center of the side of the basket, illustrating the brightness of the area as measured by the third camera. [Figure 8D] Figure 8D shows an example where the camera and lighting unit are placed in the center of the side of the basket, illustrating the brightness of the area as seen by the fourth camera. [Figure 9] Figure 9 shows the exposure timing of each camera when a product is placed in area A, in an example where the camera and lighting unit are positioned in the center of the side of the basket. [Figure 10] Figure 10 shows the exposure timing of each camera when a product is placed in area A, in an example where cameras and lighting units are placed at the four corners of the basket in a modified configuration. [Modes for carrying out the invention]

[0009] The following describes an embodiment of a cart POS system equipped with an item registration device, using drawings. The item registration device of this embodiment is composed of an information terminal, a camera which is an imaging device, and a lighting unit which is a lighting device, and is incorporated into the cart POS system. In the following description, the identification codes affixed to or printed on products are assumed to be barcodes and two-dimensional barcodes (QR code®). Furthermore, the act of photographing a product with the camera which is an imaging device is referred to as photography or exposure, the shutter open time or the time during which light is shone on the image sensor is referred to as exposure time, and the amount of light incident on the camera at that time is referred to as exposure amount. The brightness (or luminance value) of an area is considered to be synonymous with the brightness (exposure amount) of the light image (image) of the product incident on the camera when the product is placed in an illuminated area and photographed by the camera.

[0010] [First Embodiment] FIG. 1 is an overall configuration diagram of a cart POS system 1 according to a first embodiment. The cart POS system 1 includes a store server 2, a cart controller 3, an access point (shown as AP in FIG. 1) 4, and a cash register 5. The store server 2, the cart controller 3, the access point 4, and the cash register 5 are connected to a network 6 such as a LAN (Local Area Network) or a wireless LAN. The cart POS system 1 includes a plurality of information terminals 7 connected to the network 6 by wireless communication via the access point 4.

[0011] The article registration device 100 is a device that enables a customer, who is a purchaser, to self-register the products to be purchased by using the information terminal 7 and the camera 8. The information terminal 7, the camera 8 (corresponding to 8a to 8d in FIG. 3), and the lighting unit 9 (corresponding to 9a to 9d in FIG. 3) are provided on a plurality of shopping carts 10 used when a customer shops in the store. Hereinafter, the shopping cart 10 is simply referred to as the cart 10. The plurality of carts 10 are an example of a carrier for transporting products purchased by the customer. Although not shown, each cart 10 is equipped with a rechargeable battery for driving the information terminal 7, the camera 8, and the lighting unit 9.

[0012] The information terminal 7 is an information processing device that functions as a user interface for operations related to product registration or cancellation of registration (hereinafter simply referred to as "cancellation"). The information terminal 7 includes a wireless unit as a communication means with the access point 4. The access point 4 relays communication between each device connected to the network 6, that is, the store server 2, the cart controller 3, and the cash register 5, and the information terminal 7. Although only one access point 4 is shown in FIG. 1, there may be two or more depending on the scale of the store or the like.

[0013] The cash register 5 is a device that enables a store clerk or a customer to perform accounting (settlement) of the registered products. For example, when a store clerk performs accounting, a well-known face-to-face POS terminal is used, and when a customer performs accounting, a well-known self-service POS terminal is used. The number of cash registers 5 is not particularly limited.

[0014] As store operations, the store server 2 supports, for example, accounting operations, purchasing operations, inventory management operations, sales operations, and customer service operations (customer response, claim processing, etc.). To provide such support, the store server 2 manages a product database (DB) 11, a member database (DB) 12, and the like.

[0015] The product database 11 stores product records that describe product data for each product sold in the store. The product records include a product code, a price, a product name, and the like. The product code is an identification code for the product. The product database 11 includes a sales database. The sales database stores sales records that describe sales data for each item such as product, department, time zone, date, week, month, etc. The sales data includes the number of sales, the sales amount, and the like.

[0016] Also, in the present embodiment, the product database 11 stores pre-learned learning data 13 for recognizing products stored in the product database 11 by AI (Artificial Intelligence). This learning data 13 is a set of per-product learning data regarding information on each product sold in the store (including the contents, external shape, package wrapping pattern, product name (character form) described, etc.), and each per-product learning data is incorporated into the learning data 13 in association with, for example, a product code. The learning data 13 can be updated for each product-related per-product learning data in accordance with an update of the product database 11 in response to addition or replacement of products sold in the store. The per-product learning data for each product may be created by the provider of the product and provided to the store, or may be created independently in the store.

[0017] The member database 12 stores member records that describe member data for each member. Member data includes member ID, name, gender, email address, etc. The member ID is a unique code, such as a member number, assigned to each member to identify them individually. Members possess a recording medium on which their member ID is recorded. Recording mediums include, for example, magnetic cards, contact IC (Integrated Circuit) cards, contactless IC cards, smartphones, etc.

[0018] Although not specifically illustrated, store server 2 may also manage a member payment database. The member payment database stores member payment records for each member, which contain the data necessary for payment. In addition to the member ID, the member payment record includes an electronic money identification code and an electronic money user ID as data necessary for payment.

[0019] The cart POS system 1 according to this embodiment also allows for payments that do not involve the member using the payment machine 5, i.e., payments that do not involve cash payment. However, the payment method is limited to one type of electronic money. The electronic money identification code is the identification code of the electronic money that the member, identified by the member ID, has registered in advance to use for payment. The electronic money user ID is identification information used by the electronic money center EMC to identify the user of the electronic money, and is assigned by the electronic money center EMC.

[0020] To enable payments using electronic money, members register their electronic money identification code and electronic money ID in advance, for example, when registering as a member. If this prior registration for electronic money has not been performed, "NULL" or "0000" will be stored in the electronic money identification code and electronic money ID in the accounting settlement record. Furthermore, when electronic money is enabled in this manner, the store server 2 is equipped with the functionality of a communication server, or the cart POS system 1 includes a server. The store server 2 or communication server supports data communication processing between the accounting machine 5 or cart controller 3 and the electronic money center that supports electronic money payments, via an external communication network such as the Internet or a dedicated electronic money network.

[0021] The cart controller 3 works in cooperation with the information terminal 7 to support the information terminal 7 in functioning as a POS terminal. The cart controller 3 has an information terminal table 14 corresponding to each information terminal 7. In this embodiment, the information terminal table 14 is described as being provided in the cart controller 3, but a storage area may also be provided in the auxiliary storage device 23 of the information terminal 7, and the information terminal table 14 may be set for each of them. In such a case, the store server 2 may manage the information terminal table 14 together with the information terminal 7. The information terminal table 14 is part of the components of the item registration device 100.

[0022] Figure 2 is a schematic diagram showing the main data structure of the information terminal table 14. The information terminal table 14 has a terminal ID area 14a, a member ID area 14b, one or more purchased product data areas 14c, and a subtotal area 14d. The terminal ID area 14a stores the terminal ID, which is a code such as a terminal number that uniquely identifies the information terminal 7 to which the information terminal table 14 corresponds. The member ID area 14b stores the member ID. The purchased product data area 14c creates a purchased product list each time a customer registers a product they intend to purchase on the information terminal 7, that is, it stores purchased product data related to the purchased product.

[0023] Alternatively, the purchase list is updated each time the customer decides not to purchase a product registered on the information terminal 7 and cancels the registration. The purchase list consists of the product code, unit price, number of items sold, product name, sales amount, tax amount, etc., of the products the customer purchases. The subtotal area 14d is monetary data that shows the total sales amount and tax amount for all products purchased by the customer, calculated by aggregating the purchase list stored in the purchase data area 14c. This monetary data is updated each time the customer registers or cancels a product they intend to purchase, similar to the purchase list.

[0024] Figure 3 is a block diagram showing the main circuit configuration of the information terminal 7. As shown in Figure 3, the information terminal 7 comprises a processor 21, main memory 22, auxiliary storage device 23, wireless unit 26, touch panel 27, speaker 28, and system transmission line 29. The information terminal 7 may be a dedicated device or a general-purpose device such as a tablet computer. The system transmission line 29 includes an address bus, data bus, control signal lines, power supply lines for driving, etc.

[0025] The information terminal 7 connects the processor 21, main memory 22, auxiliary storage device 23, wireless unit 26, touch panel 27, and speaker 28 to the system transmission line 29. Furthermore, the information terminal 7 is connected to the reader 16, cameras 8a-8d, lighting units 9a-9d, warning light 17, and position detection unit 18 via the system transmission line 29 through interfaces (not shown).

[0026] The information terminal 7 is a computer composed of a processor 21, main memory 22, auxiliary storage device 23, and a system transmission line 29 connecting them. The processor 21 corresponds to the central part of the computer described above. The processor 21 controls various parts to realize various processes, including product registration and cancellation processing, package recognition processing and identification code recognition processing, as an information terminal 7, according to the operating system or application program. The processor 21 controls the camera's shooting timing (exposure timing) using a trigger signal, which is an arbitrarily set drive start pulse signal.

[0027] The processor 21 switches the shooting mode of cameras 8a to 8d from the continuous image mode (described later) for package recognition to the identification code close-up mode (described later) for identification code recognition. Conversely, when the application program switches back from identification code recognition processing to package recognition processing, the processor 21 also switches the shooting mode of cameras 8a to 8d back from the identification code close-up mode to the continuous image mode.

[0028] In the embodiment, the continuous image mode for package recognition is a shooting mode (video mode) that continuously captures one still image or one frame at a time. The identification code close-up mode for identification code recognition is a shooting mode that, compared to the continuous image mode described above, shortens the exposure time and shortens the focal length (corresponding to shortening the reading depth of field (DOF) in a barcode reader).

[0029] This processor 21 uses, for example, a CPU (Central Processing Unit) as its device, but is not limited to this. The processor 21 may also be multi-core / multi-threaded, capable of executing multiple processes in parallel. Furthermore, the processor 21 may incorporate an AI chip specialized for AI. In addition, the processor 21 may be implemented in a variety of other forms, including integrated circuits such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (field-programmable gate array), and DSP (Digital Signal processor).

[0030] Main memory 22 corresponds to the main memory portion of the computer described above. Main memory 22 includes a non-volatile memory area and a volatile memory area. In the non-volatile memory area of ​​main memory 22, at least the operating system or application programs are stored. Main memory 22 may also store data necessary for the processor 21 to perform processing to control each part in the non-volatile or volatile memory area. Main memory 22 uses the volatile memory area as a work area where data is rewritten as appropriate by the processor 21. The non-volatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0031] The auxiliary storage device 23 corresponds to the auxiliary storage portion of the computer described above. For example, EEPROM (Electric Erasable Programmable Read-Only Memory), HDD (Hard Disk Drive), or SSD (Solid State Drive) can be the auxiliary storage device 23. The auxiliary storage device 23 stores data used by the processor 21 in performing various processes, or data created by the processing performed by the processor 21.

[0032] The auxiliary storage device 23 may also store application programs that perform the above functions and processes. In this embodiment, the terminal program 24 is stored as the application program for performing the functions of item registration according to the embodiment (item registration and cancellation function, package recognition processing, and identification code recognition processing) for the information terminal 7, cameras 8a to 8d, and lighting units 9a to 9d. The terminal program 24 may be stored in the main memory 22.

[0033] Furthermore, the auxiliary storage device 23 stores training data 25 for each product sold at the store, which is stored in the product database 11 of the store server 2, for AI-based product recognition. This training data 25 may be a copy of the training data 13 contained in the product database 11 of the store server 2. The timing of copying this training data 13 as training data 25 may be a specific timing, such as each time the information terminal 7 is started up, or it may be at any timing, such as when the training data 25 is updated and distributed from the store server 2 to each information terminal 7.

[0034] The wireless unit 26 transmits data wirelessly to the access point 4 in accordance with the wireless communication protocol. The touch panel 27 is a device that combines the input device and display device of the information terminal 7. The touch panel 27 displays an image based on image data output from the processor 21, and also detects the touch position on the displayed image and outputs the touch position information to the processor 21. As shown in Figure 4, the information terminal 7 is mounted on the cart 10 in an orientation and position that allows the customer to visually see the display of the touch panel 27 and easily perform touch operations.

[0035] The speaker 28 is housed within the casing of the information terminal 7 and plays sounds based on sound data output from the processor 21. The sounds played include voice guidance, confirmation sounds, alert sounds, etc. The voice guidance, for example, instructs the system to read the product's identification code and perform identification code recognition if product package recognition fails. It may also emit voice information related to calling a store employee (such as a notification that an employee is being called, or the voice of the employee who responded).

[0036] The reader 16 reads the data recorded on the recording medium and outputs the read data to the processor 21. If the recording medium is a magnetic card, the reader 16 is a magnetic card reader; if it is a contact-type IC card, the reader 16 is an IC card reader. In the case of a contactless IC card or a recording medium using RFID (Radio Frequency Identification), such as a smartphone, an RFID reader is used as the reader 16. The mounting position of the reader 16 on the cart 10 is not specifically defined. For example, as shown in Figure 4, it can be mounted adjacent to the information terminal 7.

[0037] The position detection unit 18 detects the position where the product is placed (20 areas, Area A to Area T, as shown in Figures 5A to 5D and 8A to 8D). The position detection unit 18 may be composed of sensor elements such as distance sensors, or it may be calculated from image data, for example, by using parallax for the same product in images taken by cameras 8 at different, non-facing positions, and calculating the position of the product through calculation processing (triangulation, etc.). If the position of the product is calculated by calculation processing, the processor 21 may perform that processing. In that case, the position detection unit 18 is provided within the processor 21.

[0038] As shown in Figure 4, the imaging device, cameras 8a to 8d, is installed on the cart 10 so that it can photograph the inside of the basket 19 of the cart 10 from the four upper corners. In this example, one camera 8 (8a to 8d) and two illumination units 9 (9a to 9d) are combined into one imaging unit 15 (15a to 15d), with one imaging unit positioned at each corner to surround the square edge of the top of the basket 19. The processor 21 controls the operation of these cameras 8a to 8d and illumination units 9a to 9d.

[0039] In this configuration example, LED (Light Emitting Diode) lamps, which serve as the light sources for the illumination units 9a to 9d, are provided on both sides of the cameras 8a to 8d. These illumination units 9a to 9d are not constantly lit, but are illuminated when package recognition and identification code recognition are performed using the cameras 8a to 8d. While this example shows two illumination units 9a to 9d positioned on both sides of the cameras 8a to 8d, a configuration with only one illumination unit 9a to 9d on either side of the cameras 8a to 8d is also possible.

[0040] Furthermore, while this embodiment shows an example where the lighting units 9a to 9d are positioned laterally adjacent to the cameras 8a to 8d, the position of the lighting units 9a to 9d is not limited as long as the shooting area (areas A to T shown in Figure 5A, etc.) can be illuminated from the cameras 8a to 8d. For example, they may be positioned above or below the cameras 8a to 8d at the opening of the basket 19. Also, in the example shown in Figure 4, the thickness of the upper edge of the basket 19 is increased, and the protruding parts are housed within the upper edge to eliminate any obstructions. This is designed to prevent collisions between products and the cameras 8a to 8d and the lighting units 9a to 9d when products are placed in or taken out of the basket 19. This embodiment describes an example configuration in which four cameras are placed at the four corners, but of course, it is not limited to four cameras, and it is possible to implement this embodiment using one or more cameras.

[0041] In this embodiment, the image used for recognition of the identification code needs to have the exposure time as short as possible so that blurring does not occur in the captured bar. However, in order to take a bright image by shortening the exposure time, lighting is required to increase the exposure amount of the subject. In this embodiment, the illumination light from the lighting units 9a to 9d is set to light up for an exposure period based on a predetermined exposure amount when a product is placed in any area of ​​the basket 19, for example, using LED lamps. This exposure amount is such that package recognition processing and identification code recognition processing can be performed properly and no overexposure occurs. Furthermore, this exposure amount is set using the brightness of the area with the lowest brightness value, described later, as the reference value, and the exposure time is set accordingly. In response to this illumination, the cameras 8a to 8d take pictures with an exposure time that acquires the exposure amounts 31a to 31d, etc., described later. By controlling the light emission of the lighting units 9a to 9d by the processor 21, the power consumption of the battery mounted on the cart 10 can be suppressed.

[0042] Furthermore, the information terminal 7 may be equipped with a warning light 17, as shown by the dashed line in Figure 3. The warning light 17 is connected to the processor 21 via a system transmission path 29 through an interface (not shown). The warning light 17 is controlled by the processor 21 and is a lamp that notifies customers using the cart 10 and store staff of the location of customers who require attention and issues a warning.

[0043] Next, it is determined whether the customer has placed or removed items from the basket 19 as follows: Cameras 8a to 8d take continuous images or burst shots of items being placed in or removed from the basket 19 of the cart 10. Here, the latest image data of the captured continuous images is compared with the older image data stored immediately before it to detect changes in the image (composition) of the items.

[0044] If there is a change in the captured image, it is necessary to determine whether the product has been placed in or removed from the basket 19 based on the direction in which it moves within the camera 8's shooting area. For example, if it is determined that the product crossed the shooting area in a downward direction, approaching the bottom surface of the basket 19, it is determined that the product has been "placed in". On the other hand, if it is determined that the product crossed the shooting area in a direction away from the bottom surface, upward, it is determined that the product has been "removed". If the product does not cross the shooting area in the vertical direction, but moves a short distance diagonally downward or horizontally, it is considered that the product has simply shifted position within the basket 19, and no change is observed in the detected image. Furthermore, in addition to determining the direction of product movement, a weighing scale (not shown) may be placed on the bottom of the basket 19, and the placement or removal of the product may be detected by changes in weight.

[0045] Furthermore, when an item is placed in or removed from the basket 19, the processor 21 uses AI-based package recognition processing to recognize the package of the item that was placed in or removed. Specifically, the processor 21 sets a rectangular area called a bounding box for the most recently moved item in the image, and based on the learning data 25 stored in the auxiliary storage device 23, performs image recognition of the item within the bounding box to recognize which item it is.

[0046] Next, when a product is recognized by the package recognition process, the processor 21 updates the purchase product list in the information terminal table 14 by registering or deregistering the recognized product. Specifically, the information terminal 7 sends a product registration notification to the cart controller 3. Upon receiving the product registration notification, the cart controller 3 updates the purchase product list by adding or deleting the purchase product data related to the recognized product in the purchase product list in the purchase product data area 14c of the information terminal table 14, which is associated with the member ID of the information terminal 7. The purchase product list is updated each time a customer registers or cancels a product they intend to purchase using the information terminal 7.

[0047] On the other hand, if the processor 21 is unable to recognize the package of the product that has been inserted or removed, it switches to the identification code recognition process and takes a picture of the product's identification code. The identification code contained in the captured image is analyzed to recognize the product. If the product is recognized through this identification code recognition process, the purchased product list in the information terminal table 14 is updated as described above.

[0048] Next, when the customer touches the payment button on the product registration screen of the information terminal 7, the payment processing is performed by the payment machine. As part of the accounting process, the accounting machine 5, for example, the processor 21 generates a barcode based on the terminal ID assigned to the information terminal 7 and displays the barcode on the touch panel 27. The accounting machine 5 reads the barcode and sends a transaction inquiry command including the terminal ID to the cart controller 3. The cart controller 3 generates settlement data based on the list of purchased items registered in the purchased item data area 14c of the information terminal table 14 corresponding to the terminal ID included in the transaction inquiry command, and sends the settlement data back to the accounting machine 5, the source of the transaction inquiry command. Based on the received settlement data, the accounting machine 5 prompts the customer to make a payment and settles the payment for the purchased items.

[0049] Referring to Figures 5A to 5D, we will explain the brightness of the area where the product is placed for each camera in an example where cameras and lighting units are placed at the four corners of the basket 19. First, Figure 5A shows the brightness (luminance value) of an area as seen from camera 8a under illumination light when cameras 8a-8d and lighting units 9a-9d are placed at the four corners of the basket 19 and all lighting units 9a-9d are lit. Here, the brightness of an area also means the amount of exposure incident on the camera per unit time when the camera 8a takes a picture. As shown in the figure, the bottom surface of the basket 19 into which the goods are placed is divided into, for example, 20 areas A-T. The number of these divisions is arbitrary and may be set appropriately depending on the size of the bottom surface, etc.

[0050] When the brightness of the illumination in each area as seen from camera 8a is divided into five levels from "1" (dark) to "5" (bright), areas A to T will have the luminance distribution (brightness per unit area as seen from above) or light intensity distribution (amount of light per unit area) as shown in Figure 5A. Note that the brightness when all illumination units 9a to 9d are turned off is set to "0".

[0051] The brightness of each area A to T is proportional to the distance from camera 8a, with the brightness decreasing as the distance increases. Specifically, relative to camera 8a, area A is level "5", areas B, F, G are level "4", areas C, H, K, L, M are level "3", areas D, I, N, P, Q, R, S are level "2", and areas E, J, O, T are level "1".

[0052] Similarly, under illumination conditions with all lighting units 9a to 9d turned on, Figure 5B shows the brightness of each area A to T as seen from camera 8b, Figure 5C shows the brightness of each area A to T as seen from camera 8c, and Figure 5D shows the brightness of each area A to T as seen from camera 8d.

[0053] Here, we will explain the amount of exposure incident on each camera 8a to 8d from area A when camera 8 is shooting with the same exposure time. The amount of exposure incident on each camera 8a to 8d is proportional to the luminance value, which is the brightness of the area. Therefore, in Figures 5A to 5D, the level of area A is "5" for camera 8a, "2" for camera 8b, and "1" for cameras 8c and 8d. To make the brightness of the captured images at the same level for these cameras 8a to 8d, the same amount of exposure is required. Therefore, since the amount of illumination light emitted by all illumination units 9a to 9d cannot be increased or decreased linearly, it is adjusted by the exposure time of the cameras. That is, under illumination with all illumination units 9a to 9d lit, the reference exposure time is set so that the exposure amount of cameras 8c and 8d, which are at the lowest level "1", is appropriate, and the exposure times of cameras 8a and 8b at the other levels are adjusted. As a result, camera 8a needs to perform exposure control to shorten the exposure time so that the amount of incident light is reduced by four steps. Similarly, camera 8b needs to perform exposure control that shortens the exposure time by three stops.

[0054] Next, with reference to Figures 5A to 5D and Figure 6, we will explain the exposure control in cameras 8a to 8d when the purchased goods are placed in area A. Figure 6 shows the exposure timing and exposure amount of each camera 8a to 8d when photographing area A under illumination light with all illumination units 9a to 9d lit. Note that the exposure amounts 31a to 31d shown in Figure 6 are the same, although the incidence period differs because the amount of light per unit time incident on cameras 8a to 8d is different.

[0055] In this embodiment, the exposure timing (start time of shutter aperture or start time of light incidence on the image sensor) of each camera 8a to 8d is initiated by the input of a trigger signal output by the processor 21. The exposure time (shutter aperture period or image sensor light reception period) of each camera 8a to 8d is set to a fixed time common to all cameras at the time of manufacture or shipment. Of course, it is also possible to change the exposure time by changing the settings of the application program in the processor 21.

[0056] All illumination units 9a to 9d are controlled by the processor 21 to be turned on and off simultaneously. Of course, the processor 21 can also turn on and off illumination units 9a to 9d individually. As mentioned above, the illumination time (illumination period) controlled by the processor 21 is set so that, with all illumination units 9a to 9d illuminated, the exposure amount of cameras 8c and 8d at level "1" becomes the exposure amount 31c and 31d that allows for proper package recognition and identification code recognition.

[0057] Therefore, the exposure time for cameras 8c and 8d and the illumination time for all illumination units 9a to 9d are the same. In addition, for camera 8a, as mentioned above, the exposure time (exposure period) is shortened to 1 / 5 in order to reduce the incident exposure amount 31a so that the brightness of the image drops by 4 levels compared to the exposure amounts 31c and 31d of cameras 8c and 8d. Note that this 1 / 5 time reduction is the time required to reduce the brightness by 4 levels, and if the exposure time and the decrease in brightness are not in a linear proportional relationship, the exposure time will not necessarily be 1 / 5 when the brightness is reduced by 4 levels. Therefore, the processor 21 delays the trigger signal for starting the exposure of camera 8a by 4 / 5 hours relative to the exposure start timing of cameras 8c and 8d, which is the illumination start time of the illumination unit 9, and inputs it to camera 8a. In other words, the exposure start time is delayed by 4 / 5 hours from the illumination start time of the illumination unit 9. Similarly, camera 8b inputs a trigger signal to camera 8b with a 1 / 5 time delay relative to the exposure start timing of cameras 8c and 8d, in order to reduce the brightness by one stop.

[0058] Next, we will explain the exposure control in cameras 8a to 8d when the items placed in basket 19 span multiple areas. Here, for example, the center of all areas A to T is used as the reference point, and in Figure 5A, the center of the junction between area H and area M is used as the reference point. Alternatively, the diagonal line crossing the front of the camera may be used as the reference line. The positions of these reference points and reference lines can be set to any position.

[0059] From the perspective of each camera 8a to 8d, if the location of the product placed in the basket 19 is in an area farther than the reference point, the brighter area is selected; if the location of the product is in an area closer to the reference point, the darker area is selected. For example, if the product is placed in an area that straddles area A and area B, camera 8a will select the darker area B because it is closer than the reference point. Similarly, camera 8b will select either area A or area B because area A or area B has the same brightness. Furthermore, camera 8c will select the brighter area B because it is farther than the reference point, and similarly, camera 8d will select area B. Of course, this setting method is just one example, and other setting methods are also possible. For example, it is possible to use the proportion of products that straddle multiple areas as a criterion and select the brightness of the area on which more than half of the product is located.

[0060] In this way, by delaying the timing of the camera's exposure start according to the difference in brightness (luminance) of the lighting in each area within the basket 19, the amount of exposure received by each camera 8a to 8d can be made the same, and the brightness of the captured image can be made uniform at a level that is recognizable. This also prevents overexposure and blown-out highlights. Furthermore, compared to conventional automatic exposure time adjustments, the exposure time can be adjusted by shortening the time through delayed input of the trigger signal, thereby achieving faster registration processing of purchased items.

[0061] [Second Embodiment] Next, a second embodiment will be described with reference to Figures 7 and 8A to 8D. The cart POS system 1 according to this embodiment differs in the arrangement of the cameras 8a to 8d and the lighting units 9a to 9d attached to the basket 19 of the cart 10, but otherwise it is configured the same as the first embodiment described above. In the following description, the detailed explanation will be simplified by using the components and reference numerals of the first embodiment described above, and this embodiment will be described.

[0062] Figure 7 shows that in this embodiment, four cameras 8a to 8d, which are imaging devices, are provided on the cart 10 so that they can photograph the inside of the basket 19 of the cart 10 from all four sides above. In this example, one camera 8 (8a to 8d) and two illumination units 9 (9a to 9d) are combined into one imaging unit 15 (15a to 15d), and one imaging unit is positioned at the center of each of the four sides of the top of the basket 19. The processor 21 controls the operation of these cameras 8a to 8d and illumination units 9a to 9d.

[0063] Referring to Figures 8A to 8D, the brightness of the area where products are placed for each camera 8a to 8d in an example where cameras 8a to 8d and lighting units 9a to 9d are positioned at the center of each of the four sides of the basket 19 will be explained. As shown in the figure, the bottom surface of the basket 19 into which products are placed is divided into, for example, 20 areas A to T. The number of these divisions is arbitrary and may be set appropriately depending on the size of the bottom surface, etc.

[0064] First, Figure 8A shows an example where cameras 8a-8d and lighting units 9a-9d are positioned in the center of the sides of the basket 19. It shows the brightness (luminance value) of the area as seen from camera 8a, which is positioned in the center of the long side, under illumination when all lighting units 9a-9d are lit. Here, the brightness of the area also refers to the amount of exposure incident on the camera per unit time when camera 8a is taking a picture. If the brightness of the illumination light as seen from camera 8a is divided into five levels from "1" (dark) to "5" (bright), areas A-T will have the luminance distribution (brightness per unit area as seen from above) or light intensity distribution (amount of light per unit area) as shown in Figure 8A. Note that the brightness when all lighting units 9a-9d are turned off is set to "0".

[0065] The brightness of each area A to T is proportional to the distance from camera 8a, with the brightness decreasing as the distance increases. Specifically, relative to camera 8a, area C is level "5", areas B, D, GH, I, J are level "4", areas A, E, F, J, K, L, M, N, O are level "3", and areas P, Q, R, S, T are level "2".

[0066] Figure 8B shows an example where cameras 8a-8d and lighting units 9a-9d are positioned in the center of the sides of the basket 19. Under illumination with all lighting units 9a-9d lit, the figure shows the brightness (luminance value) of the area as seen from camera 8b, which is positioned in the center of the short side. Specifically, with respect to camera 8b, areas F and K are at level "5", areas A, G, L, and P are at level "4", areas B, H, M, and Q are at level "3", areas C, I, N, and R are at level "2", and areas D, E, J, O, S, and T are at level "1".

[0067] Similarly, under illumination conditions with all lighting units 9a to 9d lit, Figure 8C shows the brightness of each area A to T as seen from camera 8c, and Figure 8D shows the brightness of each area A to T as seen from camera 8d.

[0068] Next, we will explain the amount of exposure incident on each camera 8a to 8d from area A when each camera 8a to 8d has the same exposure time. In area A of Figures 8A-8D, camera 8a is at level "3", camera 8b is at level "4", camera 8c is at level "2", and camera 8d is at level "1". To make the brightness (exposure) of these cameras 8a to 8d the same level, the exposure time is adjusted as described above. That is, under lighting conditions with all illumination units 9a-9d lit, the reference exposure time is set so that the exposure of camera 8d, which is at the lowest level "1", is appropriate, and the exposure times of the other levels of cameras 8a, 8b, and 8c are adjusted accordingly. As a result, camera 8a needs to perform exposure control to shorten its exposure time so that the amount of incident light is reduced by 3 levels. Similarly, camera 8b needs to perform exposure control to shorten its exposure time so that it is reduced by 2 levels, and camera 8c needs to shorten its exposure time so that it is reduced by 1 level.

[0069] Next, referring to Figures 8A and 9, we will explain the exposure control in cameras 8a to 8d when the purchased goods are placed in area A. Figure 9 shows the exposure timing and exposure amount of each camera 8a to 8d when photographing area A under illumination with all lighting units 9a to 9d lit. Note that the exposure amounts 32a to 32d shown in Figure 9 are the same, although the incidence period differs because the amount of light per unit time incident on cameras 8a to 8d is different.

[0070] In this embodiment, the exposure timing (start time of shutter aperture or start time of light incidence on the image sensor) of each camera 8a to 8d is initiated by the input of a trigger signal output by the processor 21. The exposure time (shutter aperture period or image sensor light reception period) of each camera 8a to 8d is set to a fixed time common to all cameras at the time of manufacture or shipment. Of course, it is also possible to change the exposure time by changing the settings of the application program in the processor 21.

[0071] All illumination units 9a to 9d are controlled by the processor 21 to simultaneously turn on and off. As mentioned above, the illumination time (illumination period) controlled by the processor 21 is set so that, with all illumination units 9a to 9d illuminated, the exposure amount of camera 8d at level "1" is sufficient to properly perform package recognition and identification code recognition. Therefore, the exposure time of camera 8d and the illumination time of all illumination units 9a to 9d are the same.

[0072] In camera 8a, as mentioned above, the exposure time needs to be shortened to 3 / 5 in order to reduce the incident exposure amount 32a so that the image brightness drops by two levels compared to the exposure amount 32d of camera 8d. Therefore, the processor 21 inputs the trigger signal to start the exposure of camera 8a to camera 8a with a delay of 2 / 5 time compared to camera 8d. Note that this 2 / 5 time reduction is the time required to reduce the brightness by three levels, and if the exposure time and brightness attenuation are not in a linear proportional relationship, the exposure time will not necessarily be 2 / 5 when the brightness is reduced by three levels.

[0073] Additionally, camera 8b receives a trigger signal to reduce the brightness by three steps, with a 3 / 5 time delay relative to camera 8d. Camera 8c receives a trigger signal to reduce the brightness by one step, with a 1 / 5 time delay relative to camera 8d.

[0074] In this way, by delaying the timing of the camera's exposure start according to the difference in brightness (luminance) of the illumination light in each area within the cage 19, the amount of exposure received by each camera 8a to 8d can be made the same, and the brightness of the captured image can be made uniform at a level that is recognizable. This also prevents overexposure and blown-out highlights.

[0075] Next, a modified example will be described with reference to Figure 10. Figure 10 shows the exposure timing of each camera when a product is placed in area A, in an example where cameras 8a-8d and lighting units 9a-9d are arranged at the four corners of the basket 19 in a modified example. In the first and second embodiments described above, the exposure time of the cameras was delayed in order to reduce the amount of incident exposure so that the brightness of the image decreases relative to the exposure amount of the cameras.

[0076] In this modified version, all lighting units 9a to 9d are turned on, and shooting begins at an exposure timing where the exposure of each camera 8a to 8d starts simultaneously. After an exposure time corresponding to the brightness of the area for each camera has elapsed, the exposure is terminated. In this modified example, changing the camera's exposure time can be achieved by changing the register value that sets the exposure time of the image sensor. For example, as shown in Figure 5A above, the exposure time for each area A to T in each camera is calculated according to the brightness under illumination. An exposure time table is created in which the register value corresponding to that exposure time is set for each area of ​​each camera, and this table is stored in a rewritable format in the auxiliary storage device 23 of the information terminal 7.

[0077] The processor 21 reads the register value of the corresponding area from the exposure time table in the auxiliary storage device 23 based on the area where the item to be purchased is placed, as detected by the position detection unit 18, and modifies the register value of the image sensor. After that, it takes a picture of the item with the camera and acquires an image of the item.

[0078] In Figure 10, as shown in Figure 5A above, cameras 8c and 8d have a brightness level of "1," and in order to obtain exposure levels 33c and 33d, the exposure time and the illumination time of all lighting units 9a to 9d must be the same. Camera 8a also shortens its exposure time to 1 / 5 in order to reduce the incident exposure level 33a so that the image brightness drops by 4 levels compared to the exposure levels 31c and 31d of cameras 8c and 8d. Similarly, camera 8b shortens its exposure time to 4 / 5 in order to reduce the incident exposure level 33b so that the brightness drops by 1 level. According to this modified configuration, since the exposure of each camera 8a to 8d is completed within the period when the illumination units 9a to 9d are lit, faster shooting can be performed than in the first and second embodiments described above.

[0079] Furthermore, this embodiment of the present invention is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. The original claims of this application are included below. [C1] At least one imaging device for photographing items placed in the shooting area, The aforementioned shooting area is illuminated by at least one lighting unit for a set exposure duration, The shooting area is divided into multiple areas, and a position detection unit detects the area in which the item is placed under the illumination of the lighting unit, An information terminal that, based on the brightness of the lighting corresponding to the distance between the detected area and the imaging device, sets the exposure time under the illumination of the lighting unit for each imaging device so that the set exposure amount is obtained, takes a photograph, recognizes the object in the captured image, and registers it. An article registration device equipped with the following: [C2] The aforementioned information terminal is Based on the darkest area among the detected areas, the illumination period is set according to the set exposure amount. For each imaging device, the exposure start time of the imaging device is delayed from the illumination start time of the illumination unit so that the exposure time is set, and the image is taken with the set exposure amount. The item registration device described in C1. [C3] The aforementioned item registration device is mounted on a cart equipped with a basket, The imaging device captures images in the imaging area, which includes the bottom surface of the basket into which the items are placed from the top of the opening of the basket. The position detection unit divides the bottom surface of the basket into the plurality of areas and detects the area in which the item has been placed under the illumination of the lighting unit. The item registration device described in C1. [C4] The aforementioned information terminal is A package recognition process that recognizes the item based on the package of the item contained in the captured image, and The article is recognized by any of the following processes: an identification code recognition process that analyzes the identification code of the article contained in the captured image to recognize the article; The item registration device described in C1. [C5] Multiple carts equipped with the item registration device described in C1 and functioning as POS (Point of Sales) terminals, An accounting machine that processes payment for the registered goods, A cart POS system equipped with the following features. [C6] A program that causes the processor to execute the processing performed by each part of the device described in C1 to C5. [Explanation of symbols]

[0080] 1... Cart POS system, 2... Store server, 3... Cart controller, 4... Access point, 5... Cash register, 6... Network, 7... Information terminal, 8... Camera, 9... Lighting unit, 10... Shopping cart (cart), 11... Product database, 12... Member database, 13... Learning data, 14... Information terminal table, 14a... Terminal ID area, 14b... Member ID area, 14c... Purchased product data area, 14d... Subtotal area, 15... Basket, 16... Reader, 17... Warning light, 21... Processor, 22... Main memory, 23... Auxiliary storage device, 24... Terminal program, 25... Learning data, 26... Wireless unit, 27... Touch panel, 28... Speaker, 29... System transmission line, 100... Item registration device.

Claims

1. Multiple imaging devices that photograph items placed in the shooting area, The aforementioned shooting area is illuminated by at least one lighting unit for a set exposure duration, The shooting area is divided into multiple areas, and a position detection unit detects the area in which the item is placed under the illumination of the lighting unit, An information terminal that, based on the brightness of the lighting corresponding to the distance between the detected area and each of the imaging devices, sets the exposure time under the illumination of the lighting unit for each imaging device so that the set exposure amount is achieved, takes a photograph, recognizes the object in the captured image, and registers it. It is equipped with, The aforementioned information terminal is Based on the darkest area among the detected areas, the illumination period is set according to the set exposure amount. An item registration device that delays the exposure start time of the imaging device from the illumination start time of the illumination unit so that the exposure time is set for each imaging device, and takes a photograph with the set exposure amount.

2. The aforementioned item registration device is mounted on a cart equipped with a basket, The imaging device captures images in the imaging area, which includes the bottom surface of the basket into which the items are placed from the top of the opening of the basket. The position detection unit divides the bottom surface of the basket into the plurality of areas and detects the area in which the item has been placed under the illumination of the lighting unit. The article registration device according to claim 1.

3. The aforementioned information terminal is A package recognition process that recognizes the item based on the package of the item contained in the captured image, and The article is recognized by any of the following processes: an identification code recognition process that analyzes the identification code of the article contained in the captured image to recognize the article; The article registration device according to claim 1.

4. A plurality of carts equipped with the item registration device described in claim 1 and functioning as POS (Point of Sales) terminals, An accounting machine that processes payment for the registered goods, A cart POS system equipped with the following features.

5. A program that causes a processor to perform processing by each part of the apparatus described in any one of claims 1 to 4.

Images