Commodity registration system
The product registration system addresses erroneous registrations by using an imaging section, recognition unit, and range adjustment to ensure accurate product registration based on stored product volume or number.
Patent Information
- Application Number
- JP2023083305
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-05-19
AI Technical Summary
Conventional product registration systems face issues with erroneous registration due to the imaging device capturing the inside of the storage section, leading to multiple registrations of products based on the amount stored.
A product registration system with an imaging section in the storage section, a product recognition unit, a registration unit, and a recognition range changing unit that adjusts the recognition range based on the amount of products stored to prevent erroneous registrations.
The system effectively reduces erroneous product registrations by dynamically adjusting the recognition range based on the volume or number of products stored, ensuring accurate product registration.
Smart Images

Figure 2024166893000001_ABST
Abstract
Description
[Technical field]
[0001] An embodiment of the present invention relates to a product registration system. [Background technology]
[0002] There is a technology that allows a store selling products to install an imaging device at a position where it can image a shopping cart or basket (hereinafter also referred to as a storage unit) in which products to be purchased are placed, and to identify the product from the captured image by imaging the product when it is stored in the storage unit. With this technology, a customer can register the identified product as a transaction target by performing the action of storing the product in the storage unit.
[0003] In addition, the above-mentioned imaging device is installed above the storage unit or near the edge of the opening of the storage unit, and the imaging direction is set so that the imaging range includes the inside of the storage unit so that tray-shaped products such as bento boxes and side dishes that are difficult to tilt can be imaged from above.
[0004] However, in the above-mentioned conventional configuration, since the imaging device includes the inside of the storage unit in the imaging range, depending on the amount of products stored in the storage unit, registered products may be included in the imaging range, which may result in erroneous registration of registered products again. Therefore, a technology capable of reducing the occurrence of erroneous product registration is desired. Summary of the Invention [Problem to be solved by the invention]
[0005] An object of the present invention is to provide a product registration system capable of reducing the occurrence of erroneous product registration. [Means for solving the problem]
[0006] The product registration system of the embodiment has an imaging unit that is provided in a storage unit in which products to be purchased are stored and is positioned so as to include the inside of the storage unit within its imaging range; an identification unit that recognizes products from images captured by the imaging unit and identifies the products stored in the storage unit based on the recognition results; a registration unit that registers the products identified by the identification unit as the products to be purchased; and a modification unit that changes the recognition range in the image by which the identification unit recognizes products based on the quantity of products or the image registered by the registration unit. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of a system according to an embodiment. [Diagram 2] FIG. 2 is a side view showing an example of a general configuration of the shopping cart according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the tablet terminal according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a data configuration of the product master according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a data configuration of a sales registration table according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a functional configuration of the tablet terminal according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating a relationship between an imaging range of an imaging unit and a recognition range of a commodity recognition unit according to an embodiment. [Figure 8] FIG. 8 is a diagram illustrating a relationship between an imaging range of an imaging unit and a recognition range of a commodity recognition unit according to an embodiment. [Figure 9] FIG. 9 is a diagram illustrating a relationship between an imaging range of an imaging unit and a recognition range of a commodity recognition unit according to an embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of processing performed by the tablet terminal according to the embodiment. [Figure 11] FIG. 11 is a side view showing an example of a general configuration of the shopping cart according to the first modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] The product registration device and the program according to the present embodiment will be described in detail below with reference to the accompanying drawings. In the embodiment, a terminal device (hereinafter also referred to as a cart terminal) attached to a cart (hereinafter also referred to as a shopping cart) for transporting products used in a store will be described as an example of the product registration device. Note that the invention is not limited to this application example.
[0009] Fig. 1 is a diagram showing an example of an overall configuration of a system 1 according to an embodiment. As shown in Fig. 1, the system 1 includes a tablet terminal 2, a server 4, and a payment machine 8. The server 4 and the payment machine 8 are communicatively connected via a network Na.
[0010] The network Na may be the Internet, a virtual private network (VPN), a local area network (LAN), a public communication network, a mobile communication network, or the like, either alone or in appropriate combination. As an example, the network Na may be a combination of the Internet and a mobile communication network.
[0011] In addition, a wireless access point 7 is connected to the network Na. The wireless access point 7 is connected to the tablet terminal 2 via a network Nb such as a wireless LAN. Therefore, the tablet terminal 2 can communicate with the server 4 and the payment machine 8.
[0012] The shopping cart 3 has a storage section 31 (see FIG. 2) for storing products displayed in the store. The top of the storage section 31 is open, and products are stored in the storage section 31 from the top. A customer borrows a shopping cart 3 from a store, and while moving around the store with the shopping cart 3, stores the products he or she wishes to purchase from the products displayed in the storage section 31.
[0013] The tablet terminal 2 is provided so as to be attachable to the shopping cart 3. The tablet terminal 2 recognizes products stored in the storage unit 31 from images captured by the imaging unit 34 described below, and executes sales registration processing related to the products. Here, the sales registration processing refers to processing for making the products identified by the tablet terminal 2 the target for purchase. Specifically, the sales registration processing refers to processing for registering information related to the recognized products (hereinafter also referred to as sales registration data) in the sales registration table 242 described below. For example, in the sales registration processing, sales registration data including a product code capable of identifying the product type, etc., and the number of purchases of the product, etc., are registered in the sales registration table 242.
[0014] Furthermore, when the tablet terminal 2 accepts an operation by a customer to perform a transaction on the accounting machine 8, it executes a process to transfer the sales registration data for one transaction to the accounting machine 8. For example, the tablet terminal 2 encodes and displays the sales registration data as a code symbol such as a two-dimensional code. In this case, the accounting machine 8 reads the code symbol displayed on the tablet terminal 2 to obtain the sales registration data for one transaction registered on the tablet terminal 2, and executes accounting processing based on the sales registration data.
[0015] The server 4 uses the tablet terminal 2 as a user interface terminal and performs information processing to provide a service for carrying out buying and selling transactions of goods in the store where the user interface terminal is used. The server 4 may be, for example, a store server provided in the store, or a server device provided outside the store. Note that, although the server 4 is realized by a single device in this embodiment, it may also be a cloud server (or a cloud system) realized by multiple devices connected to a network.
[0016] The server 4 is an information processing device configured as a computer equipped with a processor, a main storage device, an auxiliary storage device, etc. The server 4 stores a product master in which the names and prices of products sold in a store are registered in a database (not shown) and provides the product master to the tablet terminal 2 and the cashier 8.
[0017] The accounting machine 8 is an accounting device installed in a store. The accounting machine 8 executes payment processing for products based on sales registration data. The payment processing includes displaying the total price and tax amount of products related to one transaction, calculating and displaying change based on the deposit received from the customer, and issuing a receipt with product information and payment information (total price, deposit amount, change amount, etc.) printed on it.
[0018] Next, the shopping cart 3 will be described. Fig. 2 is a side view showing an example of the overall configuration of the shopping cart 3. The shopping cart 3 is equipped with a basket-shaped storage section 31 with an open top, rotating casters 32 arranged below the storage section 31, and a handle section 33 connected to the storage section 31 and casters 32 and held by the customer when moving the shopping cart 3.
[0019] The storage section 31 has a generally rectangular side and bottom section surrounded by a mesh-like lattice 311 with an open top, allowing the products stored in the storage section 31 to be viewed from the outside. The side and bottom sections may be formed of walls instead of the mesh-like lattice 311.
[0020] The shopping cart 3 also includes a detachable tablet terminal 2 near the handle 33. The shopping cart 3 also includes an imaging unit 34 near the edge of the opening of the storage unit 31. The imaging unit 34 is an example of an imaging unit, and is realized by a digital camera or the like. The imaging unit 34 is installed such that the imaging direction faces the opening of the storage unit 31. In other words, the imaging direction is set so that at least a part of the imaging range of the imaging unit 34 includes the inside of the storage unit 31.
[0021] The imaging unit 34 captures an image of a commodity stored in the storage unit 31 or a commodity taken out of the storage unit 31, and outputs the captured image to the tablet terminal 2. Here, the image means a still image or a video image in frame units.
[0022] Next, a description will be given of the hardware configuration of the tablet terminal 2. FIG.
[0023] As shown in FIG. 3, the tablet terminal 2 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage unit 24, a display unit 25, an operation unit 26, a communication unit 27, and a connection unit 28.
[0024] The CPU 21 is an example of a processor, and controls the overall operation of the tablet terminal 2. The ROM 22 stores various programs. The RAM 23 is a workspace in which programs and various data are expanded.
[0025] The CPU 21, the ROM 22, and the RAM 23 are connected via a bus or the like to constitute a computer-configured control unit 200. In the control unit 200, the CPU 21 operates in accordance with a program stored in the ROM 22 or the storage unit 24 and loaded in the RAM 23, thereby executing various types of processing.
[0026] The storage unit 24 has a storage medium such as a HDD or a flash memory, and maintains the stored contents even when the power is cut off. The storage unit 24 stores programs that can be executed by the CPU 21 and various setting information. The storage unit 24 also stores a product master 241, a sales registration table 242, etc.
[0027] The product master 241 is a data table for managing products sold in a store. The sales registration table 242 is a data table for managing products that have been registered as sales using the tablet terminal 2 of the store.
[0028] The display unit 25 is a display device such as an LCD (Liquid Crystal Display). The display unit 25 displays various information under the control of the CPU 21. The operation unit 26 is an input device such as a keyboard or a pointing device. The operation unit 26 outputs operation contents input via the input device to the CPU 21. The operation unit 26 may be a touch panel provided on the display unit 25.
[0029] The communication unit 27 is a wired or wireless communication interface connectable to the network Nb. The communication unit 27 is communicatively connected to the wireless access point 7 via the network Nb. The control unit 200 communicates with the server 4 and the payment machine 8 via the communication unit 27.
[0030] The connection unit 28 is an interface for connecting a peripheral device. The connection unit 28 is a wired or wireless interface conforming to standards such as USB (Universal Serial Bus) and Bluetooth (registered trademark). In this embodiment, the connection unit 28 is connected to the imaging unit 34.
[0031] FIG. 4 is a diagram showing an example of the data configuration of the product master 241. As shown in FIG. 4, the product master 241 stores product codes and product information in association with each other. The product code is an example of identification information capable of identifying the type of product sold in a store. The product code may be, for example, a JAN code. The product information is information about a product corresponding to the product code. For example, the product name, price, volume information, etc. are registered in the product information. Here, the volume information is information indicating the size of the product corresponding to the product code, and for example, information indicating the volume or capacity of the product including the package is registered.
[0032] The data configuration of the product master 241 is not limited to that shown in FIG. 4. For example, feature information indicating the external features of a product corresponding to a product code may be registered in association with the product code. Such feature information may be used as information for matching when recognizing a product from an image. The feature information may be an image showing the external appearance of the product, or may be a feature amount that quantitatively indicates the external features of the product.
[0033] In addition, in this embodiment, the tablet terminal 2 is configured to hold the product master 241, but this is not limiting, and the server 4 may be configured to hold the product master 241. In this case, each tablet terminal 2 can refer to the product master 241 held by the server 4 and handle the product master 241 in the same way as if it were held locally.
[0034] Fig. 5 is a diagram showing an example of the data configuration of the sales registration table 242. As shown in Fig. 5, the sales registration table 242 stores an item ID and a score in association with each other. The item ID is the item ID of the item registered as a sales item, and corresponds to the item ID in the item master 241. The score is the registered score of the same item registered as a sales item. For example, the score is registered as 1 by default.
[0035] The data structure of the sales registration table 242 is not limited to that shown in Fig. 5. For example, the sales registration table 242 may be configured to register product information corresponding to a product ID in association with the product ID. Also, the sales registration table 242 may be configured to register date and time information indicating the date and time of registration in association with the product ID.
[0036] In addition, in this embodiment, the tablet terminal 2 is configured to hold the sales registration table 242, but this is not limiting, and the server 4 may be configured to hold the sales registration table 242. In this case, the server 4 can manage the sales registration table 242 for each transaction by, for example, issuing a transaction ID that can identify each transaction performed on each tablet terminal 2 and holding the sales registration table 242 in association with the transaction ID.
[0037] Specifically, the server 4 notifies the tablet terminal 2 of the issued transaction ID, thereby causing the tablet terminal 2 to retain the transaction ID while the transaction of that transaction ID is being carried out. The tablet terminal 2 then transmits the product ID of the product to be sales-registered to the server 4 along with the transaction ID, and instructs the server 4 to register the product in the sales registration table 242 corresponding to that transaction ID. This allows the server 4 to centrally manage products to be sales-registered on each tablet terminal 2. Note that the tablet terminal 2 is capable of referencing the sales registration table 242 for transactions related to its own terminal.
[0038] Next, a description will be given of the functional configuration of the tablet terminal 2. FIG.
[0039] As shown in FIG. 6, the tablet terminal 2 includes an image acquisition unit 201, a commodity recognition unit 202, a registration processing unit 203, a recognition range change unit 204, and a payment processing unit 205 as functional components.
[0040] The functional configuration of the tablet terminal 2 is a software configuration that is realized by the processor of the tablet terminal 2 operating in accordance with a program stored in a ROM or memory unit, but is not limited to this. For example, some or all of the functional configuration of the tablet terminal 2 may be realized by hardware such as a dedicated circuit.
[0041] The image acquisition unit 201 acquires an image captured by the imaging unit 34 via the connection unit 28. The commodity recognition unit 202 is an example of an identification unit. The commodity recognition unit 202 identifies the commodity imaged by the imaging unit 34 based on the image acquired by the image acquisition unit 201. Specifically, when a commodity enters the imaging range of the imaging unit 34 and is imaged by the imaging unit 34, the commodity recognition unit 202 recognizes the commodity by using a known object recognition technology or the like, thereby identifying the commodity code of the commodity. The commodity recognition unit 202 may identify and decode a code symbol attached to the commodity to identify the commodity code held in the code symbol.
[0042] Here, the relationship between the imaging range of the imaging unit 34 and the recognition range in which the commodity recognition unit 202 recognizes a commodity from an image will be described with reference to Fig. 7 and Fig. 8. Here, Fig. 7 and Fig. 8 are diagrams that diagrammatically show the relationship between the imaging range of the imaging unit 34 and the recognition range of the commodity recognition unit 202.
[0043] 7, a rectangular frame area indicates the imaging range Aa. The imaging range Aa is set so as to include the inside of the storage unit 31. In other words, the image captured by the imaging unit 34 includes the inside of the storage unit 31.
[0044] In addition, in the imaging range Aa, the hatched area indicates the recognition range Ab of the commodity recognition unit 202. That is, Fig. 7 shows that the entire area of the image captured by the imaging unit 34 becomes the recognition range Ab. Note that the recognition range Ab and the imaging range Aa have an inclusive relationship (Ab ⊂ Aa).
[0045] When a customer takes out a product that he or she wishes to purchase from the display shelf and stores the product in the storage unit 31, the product G comes into the imaging range Aa of the imaging unit 34, as shown in Fig. 8. The product recognition unit 202 recognizes the product G from the recognition range Ab of the image acquired by the image acquisition unit 201, that is, from the entire area of the image.
[0046] Furthermore, when the commodity recognition unit 202 recognizes the commodity G from the images, it detects the direction in which the commodity G moves from the temporally consecutive images. Specifically, the commodity recognition unit 202 detects the movement (e.g., the motion vector) of the commodity G relative to the storage unit 31. That is, the commodity recognition unit 202 detects information that can determine whether the commodity G has been stored in the storage unit 31 or whether the commodity G has been removed from the storage unit 31. Note that the commodity recognition unit 202 identifies the commodity code once for the commodity recognized from the series of images.
[0047] Returning to FIG. 6, the registration processing unit 203 is an example of a registration unit. The registration processing unit 203 executes sales registration processing of the product identified by the product recognition unit 202. Specifically, when the product recognition unit 202 recognizes a product and detects that the product is moving downward, the registration processing unit 203 determines that the product is stored in the storage unit 31. In this case, the registration processing unit 203 registers the product code and points of the product identified by the product recognition unit 202 in the sales registration table 242. Specifically, the registration processing unit 203 adds 1 to the point of the product every time the same product is registered. In addition, the registration processing unit 203 reads out the product name and price of the product corresponding to the registered product code from the product master 241, and displays them on the display unit 25 together with the point, thereby displaying the products registered for sales in a list in the order of registration.
[0048] Furthermore, when the product recognition unit 202 recognizes a product and detects that the product is being moved to the information, the registration processing unit 203 determines that the product has been removed from the storage unit 31. In this case, the registration processing unit 203 deletes the product code of the product identified by the product recognition unit 202 from the sales registration table 242 and subtracts one from the registered points. Specifically, when the registered points become 0, the registration processing unit 203 deletes the product code of the product from the sales registration table 242. Furthermore, the registration processing unit 203 notifies the product whose sales registration has been cancelled by erasing the display of the product deleted from the sales registration table 242 from the products displayed in the list and subtracting the points.
[0049] In this way, in the system 1, when a customer stores an item in the storage unit 31, the sales registration of that item is automatically performed. Also, in the system 1, when a customer removes an item from the storage unit 31, the registration of that item is automatically canceled.
[0050] Incidentally, since the products stored in the storage unit 31 are held within the storage unit 31, the more products stored, the more space the products occupy within the storage unit 31. Also, when a large product is stored in the storage unit 31, the space the product occupies within the storage unit 31 increases. In this case, if a product stored in the storage unit 31 falls within the recognition range Ab of the product recognition unit 202, the product recognition unit 202 may recognize the product as a new product even though it is an already registered product, and the registration processing unit 203 may erroneously register the sales multiple times.
[0051] Therefore, in this embodiment, the recognition range change unit 204 performs processing to reduce the occurrence of the above-mentioned erroneous registration by changing the recognition range Ab of the product recognition unit 202 based on the amount of products stored in the storage unit 31 derived from the sales-registered products. Note that the recognition range change unit 204 is an example of a change unit.
[0052] Specifically, the recognition range change unit 204 reads out volume information of products corresponding to each product code registered in the sales registration table 242 from the product master 241. Next, the recognition range change unit 204 calculates the total volume of the products stored in the storage unit 31 by adding up the values of the read volume information. Here, the total volume of the products is an example of information representing the state of the products stored in the storage unit 31. Then, the recognition range change unit 204 changes the recognition range Ab according to the calculated total volume.
[0053] Fig. 9 is a diagram for explaining the operation of the recognition range change unit 204, and shows the relationship between the imaging range Aa of the imaging unit 34 and the recognition range Ab of the commodity recognition unit 202. Note that Fig. 9 shows a state in which two commodities Ga and Gb are stored in the storage unit 31.
[0054] In this case, the recognition range change unit 204 changes the recognition range Ab according to the total volume of the products Ga and Gb. As an example, the recognition range change unit 204 changes the recognition range Ab when the total volume exceeds a predetermined threshold. Fig. 9 shows an example in which the remaining area (the upper half of the image capture range Aa) excluding the area representing the storage section 31 from the image capture range Aa is set as the recognition range Ab.
[0055] This allows the tablet terminal 2 to exclude registered products Ga and Gb stored in the storage unit 31 from the recognition target, thereby preventing erroneous registration. Also, since the tablet terminal 2 has the area above the storage unit 31 as the recognition range Ab, it can recognize a newly stored product Gc and register the sales of the product Gc.
[0056] In the above example, the recognition range Ab is changed based on one threshold value, but the recognition range Ab may be changed (reduced or enlarged) in multiple stages by providing multiple threshold values. Also, the recognition range Ab may be dynamically changed (reduced or enlarged) in accordance with the change in the total volume.
[0057] In the above example, the recognition range Ab is changed based on the total volume of the products registered for sales, but the recognition range Ab may be changed using other indices. For example, the recognition range Ab may be changed based on the total number of products registered for sales. Here, the total number of products is an example of information indicating the state of the products stored in the storage unit 31.
[0058] In this case, too, the recognition range Ab may be changed (reduced or enlarged) based on one or more threshold values, or the recognition range Ab may be dynamically changed (reduced or enlarged) in accordance with changes in the total number.
[0059] Returning to Fig. 6, the payment processing unit 205 executes processing related to the payment of the registered sales item in response to the customer's operation. Specifically, the payment processing unit 205 inputs data capable of identifying the registered sales item into the payment machine 8, thereby enabling the payment machine 8 to be in a state in which the item can be paid for.
[0060] For example, when the payment processing unit 205 receives an operation to start payment via the operation unit 26, it encodes the sales registration data registered in the sales registration table 242 into a code symbol such as a two-dimensional code and displays it on the display unit 25. In this case, the payment machine 8 obtains the sales registration data by reading the code symbol displayed on the display unit 25 of the tablet terminal 2, and performs payment processing for the products for one transaction that have been registered as sales based on the obtained sales registration data.
[0061] Furthermore, for example, the payment processing unit 205 may transmit the sales registration data registered in the sales registration table 242 to the payment machine 8 by short-range wireless communication such as Bluetooth (registered trademark). In this case, the payment machine 8 acquires the sales registration data transmitted from the tablet terminal 2, and executes the payment process for the products for one transaction that have been registered as sales, based on the acquired sales registration data.
[0062] Also, for example, in a configuration in which the sales registration table 242 is stored in the server 4, the payment processing unit 205 encodes the transaction ID of the transaction into a code symbol such as a barcode or two-dimensional code, and displays it on the display unit 25. In this case, the payment machine 8 obtains the transaction ID by reading the code symbol displayed on the display unit 25 of the tablet terminal 2. The payment machine 8 then obtains sales registration data related to the obtained transaction ID from the server 4, and performs payment processing for the products for one transaction that have been registered as sales, based on the obtained sales registration data.
[0063] The payment for the products is not limited to the payment using the cashier 8, but may be made on the tablet terminal 2. For example, when the payment processing unit 205 receives an instruction to start payment via the operation unit 26, the payment processing unit 205 displays a screen on the display unit 25 that allows the user to select a payment method such as electronic money payment or credit card payment. When the payment method is selected, the payment processing unit 205 receives input of payment information for making the payment using the selected payment method, and executes a payment process to settle the total amount of the products registered as sales based on the input payment information by cooperating with an external payment server. Such payment information may be, for example, a credit card number or a two-dimensional code for code payment. In the case of a two-dimensional code for code payment, the input may be received via the imaging unit 34.
[0064] An example of the operation of the tablet terminal 2 will be described below with reference to Fig. 10. Fig. 10 is a flowchart showing an example of processing performed by the tablet terminal 2. Note that, as a premise of this processing, it is assumed that the recognition range Ab of the commodity recognition unit 202 is set to a range equivalent to the imaging range Aa of the imaging unit 34. Also, in this processing, an example of operation will be described in which the recognition range Ab of the commodity recognition unit 202 is changed based on one threshold value.
[0065] First, when the control unit 200 of the tablet terminal 2 starts this process in response to an operation via the operation unit 26, it initializes the sales registration table 242 (step S11).
[0066] In addition, when the sales registration table 242 is configured to be held by the server 4, the control unit 200 requests the server 4 to issue a transaction ID, thereby causing the server 4 to hold the sales registration table 242 related to the transaction ID of the transaction. In addition, the control unit 200 obtains the issued transaction ID from the server 4 and holds it in the RAM 23 or the like.
[0067] Next, the image acquisition unit 201 acquires an image from the imaging unit 34 (step S12). Next, the commodity recognition unit 202 determines whether or not the commodity is recognized from the recognition range Ab of the image acquired in step S12 (step S13). If the commodity is not recognized (step S13; No), the process proceeds to step S21.
[0068] When the commodity recognition unit 202 recognizes a commodity from the recognition range Ab of the image acquired in step S12 (step S13; Yes), it identifies the commodity code of the commodity (step S14). Next, the commodity recognition unit 202 determines the movement direction of the recognized commodity (step S15). Here, if the movement direction of the commodity is downward toward the storage unit 31 (step S15; Yes), the registration processing unit 203 registers the commodity identified in step S14 in the sales registration table 242 (step S16) and proceeds to step S18. On the other hand, if the movement direction of the commodity is upward away from the storage unit 31 (step S15; No), the registration processing unit 203 deletes the commodity identified in step S14 from the sales registration table 242 (step S17) and proceeds to step S18.
[0069] Next, the recognition range change unit 204 judges whether the total volume or the total number of products registered in the sales registration table 242 is equal to or less than a threshold value (step S18). If it is judged to be equal to or less than the threshold value (step S18; Yes), the recognition range change unit 204 sets the recognition range Ab of the product recognition unit 202 to the entire image capture range Aa (step S19) and proceeds to step S21. If it is judged to be greater than the threshold value (step S18; No), the recognition range change unit 204 sets the recognition range Ab of the product recognition unit 202 to the upper half of the image capture range Aa excluding the storage unit 31 (step S20) and proceeds to step S21.
[0070] Next, the control unit 200 judges whether or not an instruction to end the sales registration has been given (step S21). If an instruction to end the sales registration has not been given (step S21; No), the control unit 200 returns the process to step S12. If an instruction to end the sales registration has been given (step S21; Yes), the control unit 200 ends this process.
[0071] As described above, the system 1 has the imaging unit 34 that is provided in the storage unit 31 in which products to be purchased are stored and arranged so as to include the inside of the storage unit in its imaging range, the product recognition unit 202 of the tablet terminal 2 that recognizes products from images captured by the imaging unit 34 and identifies the products stored in the storage unit 31 based on the recognition results, and the registration processing unit 203 of the tablet terminal 2 that registers the identified products as products to be purchased. In addition, the recognition range change unit 204 of the tablet terminal 2 changes the recognition range of the registration processing unit 203 based on information related to the products registered by the registration processing unit 203.
[0072] As a result, in the system 1, when the amount of products stored in the storage unit 31 causes registered products to fall within the imaging range of the imaging unit 34, the recognition range change unit 204 changes the recognition range, thereby making it possible to exclude the registered products from the recognition range. Therefore, in the system 1, it is possible to reduce the occurrence of erroneous product registration.
[0073] The above-described embodiment can be modified as appropriate by changing a part of the configuration or function of each of the above-described devices. Therefore, some modified examples of the above-described embodiment will be described below as other embodiments. Note that the following mainly describes the differences from the above-described embodiment, and a detailed description of the common points with the contents already described will be omitted. The modified examples described below may be implemented individually or in appropriate combination.
[0074] (Variation 1) In the above embodiment, the number of imaging units 34 attached to the shopping cart 3 is one, but this is not limiting and multiple imaging units may be provided. For example, as shown in Fig. 11, another imaging unit 35 may be provided at a position different from that of the imaging unit 34. Fig. 11 is a side view showing an example of the overall configuration of the shopping cart 3 according to the first modification.
[0075] The imaging direction of the imaging unit 35 is directed toward the opening of the storage unit 31, similar to the imaging unit 34, and is installed so that at least a part of the imaging range includes the inside of the storage unit 31. The imaging unit 35 is also connected to the tablet terminal 2, and an image obtained by imaging is output to the tablet terminal 2.
[0076] In this case, the image acquisition unit 201 of the tablet terminal 2 acquires images captured by each of the imaging units 34 and 35. The commodity recognition unit 202 recognizes and identifies commodities from the images captured by each of the imaging units 34 and 35. The registration processing unit 203 performs sales registration of the commodities identified by the commodity recognition unit 202.
[0077] In this case, in order to prevent multiple registration of a product, the registration processing unit 203 performs sales registration of the product only when a common product code is identified from the images of the imaging units 34 and 35, or when a product code is registered from an image of either the imaging units 34 or 35. In this case, the registration processing unit 203 may be configured to register information indicating the acquisition source (imaging units 34 and 35) of the image from which the product was identified (hereinafter, acquisition source information) when registering the product. Note that, if different product codes are identified from the images of the imaging units 34 and 35, the registration processing unit 203 does not perform sales registration and causes the display unit 25 to display a message informing of an error.
[0078] The recognition range change unit 204 changes the recognition range Ab of the commodity recognition unit 202 based on the total volume or the total number of the commodities registered for sales, as in the above-described embodiment. However, since a plurality of imaging units are provided, the recognition range change unit 204 changes the recognition range Ab for each of the imaging units 34 and 35. In this case, the recognition range change unit 204 may change the recognition range Ab for each of the imaging units 34 and 35 in the same manner, or may change them individually. In the latter case, the recognition range change unit 204 calculates the total volume or the total number of the commodities registered for sales for each image acquisition source, for example, based on the acquisition source information registered in the sales registration table 242. Then, the recognition range change unit 204 changes the recognition range Ab for each of the imaging units 34 and 35 individually based on the calculation result.
[0079] As a result, in the tablet terminal 2, for example, when the products stored in the storage unit 31 are stored in a state where they are biased toward a certain part of the storage unit 31, the recognition range Ab of the imaging unit that images that part can be individually changed. Therefore, in the tablet terminal 2 according to this modification, the recognition range Ab can be set according to the storage status of the products in the storage unit 31, so that the occurrence of erroneous registration can be prevented while the recognition of products can be efficiently performed.
[0080] (Variation 2) In the above embodiment, the imaging range Aa of the imaging unit 34 is fixed, but this is not limiting, and the imaging range Aa may be configured to be movable in the vertical direction, for example. In this case, for example, the imaging unit 34 may be configured to include a drive unit such as an actuator that can move the imaging direction of the imaging unit 34 up and down, and the drive unit may be driven in response to a drive signal input from the tablet terminal 2.
[0081] In this case, the registration processing unit 203 of the tablet terminal 2 changes the imaging range Aa of the imaging unit 34, that is, the recognition range Ab, by outputting an instruction signal to the imaging unit 34 to drive the drive unit according to the total volume or total number of sales-registered products. Specifically, the registration processing unit 203 outputs a drive signal to move the imaging unit 34 upward when the total volume or total number of sales-registered products increases, such as exceeding a threshold value. This makes it possible to reduce the occurrence of erroneous registrations, similar to the above-described embodiment.
[0082] (Variation 3) In the above embodiment, the recognition range Ab of the product recognition unit 202 is changed based on the total volume or total number of products registered as sales, but this is not limiting, and the recognition range Ab may be changed based on an image captured by the imaging unit 34. Specifically, the recognition range change unit 204 recognizes the state of the products stored in the storage unit 31 from the image acquired by the image acquisition unit 201, and changes the recognition range Ab according to the range in which the recognized products exist. For example, when the recognition range change unit 204 recognizes the presence of a product below the image, it changes the recognition range Ab by excluding the range including the recognized product from the recognition range Ab. This makes it possible to reduce the occurrence of erroneous registrations, as in the above embodiment.
[0083] The recognition method used by the recognition range change unit 204 to recognize a commodity from an image may be the same as that used by the commodity recognition unit 202. Furthermore, the recognition range change unit 204 may recognize a commodity stored in the storage unit 31 based on the recognition results of the commodity recognition unit 202 and the recognition range change unit 204. For example, a commodity that is substantially stationary in successive images may be recognized as a commodity stored in the storage unit 31.
[0084] (Variation 4) In the above embodiment, the embodiment in which products are stored in the storage section 31 of the shopping cart 3 has been described, but the present invention is not limited to this, and products may be stored in other types of storage section 31. For example, a shopping basket used for carrying products in a store may be an example of the storage section 31. In this case, the imaging section 34 is installed near the edge of the opening of the shopping basket, etc., and is installed so that at least a part of the imaging range includes the inside of the shopping basket.
[0085] In this case, the tablet terminal 2 may be replaced with a mobile terminal such as a smartphone having the same functions as the tablet terminal 2. The shopping basket may be configured to be carried by the customer or may be configured to be carried in the storage section 31 of the shopping cart 3.
[0086] (Variation 5) In the above-described embodiment, the tablet terminal 2 recognizes (and identifies) the commodity from the image captured by the imaging unit 34, but this is not limiting and an external device such as the server 4 may perform this. In this case, for example, the tablet terminal 2 may transmit an image captured by the image acquisition unit 201 to the server 4, and the server 4 may include some or all of the commodity recognition unit 202, the registration processing unit 203, and the recognition range change unit 204. In this case, the server 4 may recognize (and identify) the commodity and perform sales registration processing based on the image transmitted from the tablet terminal 2.
[0087] The programs executed by each device in the above-described embodiments (and the modified examples) are provided in a state where they are pre-installed in a ROM, a storage unit, etc. The programs executed by each device in the above-described embodiments may be provided by being recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD) in the form of an installable or executable file.
[0088] Furthermore, the programs executed by each device of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by each device of the above-described embodiments may be provided or distributed via a network such as the Internet.
[0089] Although the embodiment of the present invention has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. These new embodiments and their modifications can be implemented in various other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]
[0090] 1 System 2. Tablet devices 3. Shopping Cart 4 Server 8. Cashier 200 Control section 201 Image Acquisition Unit 202 Product recognition department 203 Registration processing unit 204 Recognition range change section 205 Payment Processing Unit 241 Product Master 242 Sales registration table [Prior art documents] [Patent documents]
[0091] [Patent Document 1] JP 2019-219966 A
Claims
1. an imaging unit that is provided in a storage unit in which the product to be purchased is stored and that is disposed so as to include the inside of the storage unit in its imaging range; an identification unit that recognizes products from the images captured by the imaging unit and identifies the products to be stored in the storage unit based on the recognition result; a registration unit that registers the product identified by the identification unit as the product to be purchased; a change unit that changes a recognition range in the image in which the identification unit recognizes products based on the amount of products registered by the registration unit or the image; A product registration system having the above structure.
2. The product registration system according to claim 1 , wherein the change unit changes the recognition range based on a total volume of the products registered by the registration unit.
3. The product registration system according to claim 1 , wherein the change unit changes the recognition range based on a total number of products registered by the registration unit.
4. The product registration system according to claim 1 , wherein the change unit, when recognizing a product stored in the storage unit from the image, excludes an area including the recognized product from the recognition area.
5. The identification unit identifies the product to be taken out of the storage unit based on the recognition result, When the identification unit identifies the product removed from the storage unit, the registration unit excludes the product from the items to be purchased. The product registration system according to any one of claims 1 to 4.
Citation Information
Patent Citations
Commodity storage device, commodity registration device, and commodity registration method
JP2013041465A
Display controller and program
JP2018097462A
Commodity pickup apparatus
JP2022089455A
Automated checkout system through mobile shopping units
US20180218351A1
Article information acquisition system, shopping support system, shopping support method, and transport apparatus
JP2019219966A
