Article identification system

The product identification system addresses the need for frequent dictionary updates by using image capture, recognition, and code reading to generate error data, automating the update process and reducing administrative effort.

JP2024156465AInactive Publication Date: 2024-11-06TOSHIBA TEC KK
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023070949
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-24
Publication Date
2024-11-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing product identification systems require frequent maintenance of dictionary data due to changes in product packaging and the absence of data for newly available products, leading to a significant administrative burden.

Method used

A product identification system that includes an acquisition means for capturing product images, an identification means for recognizing products using external appearance characteristics, a reading means for decoding visible codes, and a generation means for creating error data when identification fails, allowing for automated dictionary updates based on error data.

Benefits of technology

Reduces the administrative burden of maintaining dictionary data by automatically updating the system with error data, ensuring accurate product identification and minimizing manual intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156465000001_ABST
    Figure 2024156465000001_ABST
Patent Text Reader

Abstract

To reduce a burden on a manager regarding maintenance of dictionary data used for article identification.SOLUTION: An article identification system comprises acquisition means, identification means, read means, and generation means. The acquisition means acquires an image generated by photographing an article that a purchaser puts in a container to purchase it. The identification means identifies the article based upon features of the article represented in the acquired image and dictionary data representing features of appearances of a plurality of articles. The read means reads an identifier of the article out of a visible code shown on the article. If the article cannot be identified by the identification means although the identifier is included in the acquired image, but an identifier is read for the article which cannot be identified, the generation means generates error data for managing the identification error which includes the acquired image and the read identifier.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present invention relates to an item identification system. [Background technology]

[0002] A registration system has already been proposed for display sales stores, which identifies from an image an item that a customer places in a shopping cart and automatically registers the item as an item to be purchased. However, product packages are frequently changed, and if the dictionary data referenced for identification is no longer appropriate, there is a risk that the product cannot be identified. Also, if data on new products is not included in the dictionary data, the product cannot be identified. This requires the system administrator to frequently maintain the dictionary data, which places a heavy burden on the system administrator. In view of these circumstances, it has been desired to reduce the burden on administrators in maintaining dictionary data used for product identification. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Republished Patent Publication WO2017 / 150226 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a product identification system capable of reducing the burden on an administrator regarding the maintenance of dictionary data used for product identification. [Means for solving the problem]

[0005] The product identification system of the embodiment includes an acquisition means, an identification means, a reading means, and a generating means. The acquisition means acquires an image of a product that is placed in a container by a purchaser for the purpose of purchase. The identification means identifies the product based on the product features shown in the image acquired by the acquisition means and dictionary data representing the external features of each of a plurality of products. The reading means reads an identifier of the product from a visible code displayed on the product. When a product is captured in the image acquired by the acquisition means but cannot be identified by the identification means and an identifier is read by the reading means for the product that could not be identified, the generating means generates error data for managing the identification error, the error data including the image acquired by the acquisition means and the identifier read by the reading means. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system according to an embodiment and a main circuit configuration of a cart terminal and a POS server. [Diagram 2] 2 is a perspective view showing an example of a state in which the cart terminal in FIG. 1 is attached to a cart. [Diagram 3] 13 is a flowchart of a registration process. [Figure 4] 13 is a flowchart of a registration process. [Diagram 5] 13 is a flowchart of a dictionary management process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] An example of an embodiment will be described below with reference to the drawings. Note that as this embodiment, an example of a transaction processing system implemented by incorporating the functions of a product identification system will be described. FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system 1 according to this embodiment, and the main circuit configuration of a cart terminal 100 and a POS server 500. As shown in FIG. The transaction processing system 1 is configured so that a cart terminal 100, a billing machine 200, an attendant terminal 300, a management terminal 400, and a POS server 500 can communicate with each other via a communication network 2.

[0008] The communication network 2 may be the Internet, a virtual private network (VPN), a local area network (LAN), a public communication network, a mobile communication network, or the like, either alone or in appropriate combination. As an example, the communication network 2 may be a combination of a LAN and the Internet. Although any number of cart terminals 100, accounting machines 200, attendant terminals 300, control terminals 400, and POS servers 500 may be included in transaction processing system 1, only one of each is shown in FIG.

[0009] The cart terminal 100 is an information processing terminal attached to a shopping cart (hereinafter referred to as a cart) provided in a store. The cart terminal 100 is loaned together with a cart to a purchaser who purchases a product at the store. The cart terminal 100 is a terminal device that executes a registration process for registering a product that the purchaser plans to purchase as a purchased product in response to an operation by the purchaser. As will be described later, the cart terminal 100 executes a process for identifying a product to be registered as a purchased product from an image in which the product is reflected. In other words, the cart terminal 100 has a function as a product identification device.

[0010] The payment machine 200 is installed in a store and executes payment processing related to the settlement of the price of purchased items registered by the cart terminal 100. The payment machine 200 is operated by an operator during the payment processing. The operator of the payment machine 200 is usually the purchaser. In some cases, the operator of the payment machine 200 is a store clerk.

[0011] Attendant terminal 300 is an information processing terminal operated by a store clerk. Attendant terminal 300 is a terminal device for a user interface related to information processing to support the work of the store clerk regarding a transaction processed by transaction processing system 1. The work of the store clerk is, for example, to monitor the implementation status of a transaction being processed and to support the purchaser appropriately. Management terminal 400 is an information terminal that serves as a user interface for an administrator who manages transaction processing system 1. As management terminal 400, various general-purpose information processing devices such as desktop type, notebook type, tablet type, and mobile type can be used.

[0012] The POS server 500 is an information processing device that executes management processing for managing product buying and selling transactions processed by the registration processing in the cart terminal 100 and the accounting processing in the accounting machine 200. The POS server 500 also executes information processing (hereinafter referred to as dictionary management processing) described below for managing dictionary data for product identification. The POS server 500 functions as a dictionary management device by executing the dictionary management processing.

[0013] The cart terminal 100 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, a touch panel 104, a sound unit 105, an interface unit 106, a wireless communication unit 107, and a transmission path 108. The processor 101 is capable of communicating with the main memory unit 102, the auxiliary memory unit 103, the touch panel 104, the sound unit 105, the interface unit 106, and the wireless communication unit 107 via the transmission path 108.

[0014] The processor 101, the main storage unit 102, and the auxiliary storage unit 103 are connected via a transmission line 108 to constitute a computer that performs information processing for controlling the cart terminal 100. The processor 101 corresponds to the central part of the computer. The processor 101 executes information processing for controlling each part to realize various functions of the cart terminal 100 in accordance with information processing programs such as an operating system and application programs.

[0015] The main memory unit 102 corresponds to the main memory portion of the computer. The main memory unit 102 includes a read-only memory area and a rewritable memory area. The main memory unit 102 stores a part of the information processing program in the read-only memory area. The main memory unit 102 may also store data required for the processor 101 to execute processes for controlling each part in the read-only memory area or the rewritable memory area. The main memory unit 102 uses the rewritable memory area as a work area for the processor 101.

[0016] The auxiliary storage unit 103 corresponds to the auxiliary storage part of the computer. The auxiliary storage unit 103 may be, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), an SSD (solid state drive), or any other known storage device. The auxiliary storage unit 103 stores data used by the processor 101 in performing various processes and data generated by the processes in the processor 101. The auxiliary storage unit 103 may also store the information processing program. In this embodiment, the auxiliary storage unit 103 stores a cart terminal program PRA, which is one of the information processing programs. The cart terminal program PRA is an application program in which the procedure of the registration process is described. A part of the storage area of ​​the auxiliary storage unit 103 is used as an area for storing dictionary data DAA and transaction data DAB. The dictionary data DAA is data representing an identification dictionary for identifying which product is reflected in an image. The transaction data DAB is data representing the content of one transaction. At a predetermined dictionary download timing, for example after the cart terminal 100 is started, the dictionary data DAC is downloaded from the POS server 500 to the cart terminal 100 and stored in the auxiliary storage unit 103 as dictionary data DAA.

[0017] The touch panel 104 displays a screen for presenting information to the operator, and also inputs instructions by the operator touching the screen through the touch panel 104. The sound unit 105 outputs sounds for various guidance and warnings. As the sound unit, various well-known sound devices such as a voice synthesis device and a buzzer can be used alone or in combination.

[0018] An external device such as a camera 199 is connected to the interface unit 106. The interface unit 106 interfaces for sending and receiving data to and from the connected external device. An existing USB (universal serial bus) controller or the like can be used as the interface unit 106. The camera 199 captures an image within a range that encompasses the entire internal space of a shopping basket placed on the cart. The camera 199 then outputs image data representing the image obtained by capturing the image. The camera 199 may repeat capturing images at regular intervals, or may capture images in response to an instruction from the processor 101. A camera built into the cart terminal 100 may be used instead of the camera 199.

[0019] The wireless communication unit 107 executes communication processing for wirelessly performing data communication via the communication network 2. For example, an existing wireless communication device for wireless LAN can be used as the wireless communication unit 107. Note that instead of or in addition to the wireless communication unit 107, a communication unit connected to the communication network 2 by wire may be used. The transmission path 108 includes an address bus, a data bus, and control signal lines, and transmits data and control signals between the connected components.

[0020] The basic hardware of the cart terminal 100 is assumed to be, for example, the hardware of a tablet-type information processing device. The cart terminal 100 is generally transferred in a state in which the cart terminal program PRA is stored in the auxiliary storage unit 103. However, the cart terminal program PRA may be transferred separately from the hardware in a state in which the cart terminal program PRA is not stored in the auxiliary storage unit 103, or in a state in which a different version of the same type of application program is stored in the auxiliary storage unit 103. The cart terminal 100 may be configured by writing the cart terminal program PRA to the auxiliary storage unit 103 in response to an operation by an arbitrary worker. The cart terminal program PRA may be transferred by recording it on a removable recording medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory, or by communication via a network.

[0021] FIG. 2 is a perspective view showing an example of how the cart terminal 100 is attached to the cart. The cart 900 includes a caster portion 910 , a handle frame portion 920 , a basket portion 930 , a support portion 940 , and a battery case 950 . The caster unit 910 has four wheels 911 for smoothly moving the cart 900 on a floor surface. The wheels 911 are attached to a frame 912 in a state capable of rotating around an axis in the vertical direction.

[0022] The handle frame portion 920 includes a pair of vertical frames 921, 921 and a handle bar 922. The vertical frames 921, 921 are erected above the two wheels of the caster portion 910. The handle bar 922 connects the upper ends of the vertical frames 921, 921 together. The basket section 930 is provided horizontally from the middle of the handle frame section 920. The basket section 930 is shaped like a basket with an opening at the top to allow products to be placed inside, and functions as a container for storing products that the purchaser plans to purchase. A camera 199 is attached to the upper end of the basket section 930 on the side closer to the handle bar 922 in a state in which the opening of the basket section 930 can be photographed from the side.

[0023] The support unit 940 includes a pole 941. The pole 941 is attached to one of the vertical frames 921 so that its tip is positioned higher than the handlebars 922. The cart terminal 100 described above is attached to the tip of the pole 941. As a result, the support unit 940 supports the cart terminal 100 in the state shown in FIG. The battery case 950 is attached between the vertical frames 921, 921 on the lower end side of the handle frame portion 920. The battery case 950 houses a battery used as an external power source for the cart terminal 100.

[0024] 1, the POS server 500 includes a processor 501, a main memory unit 502, an auxiliary memory unit 503, a communication unit 504, and a transmission path 505. The processor 501, the main memory unit 502, the auxiliary memory unit 503, and the communication unit 504 are capable of communicating with each other via the transmission path 505.

[0025] The processor 501 , the main memory unit 502 and the auxiliary memory unit 503 are connected via a transmission line 505 to constitute a computer that performs information processing for controlling the POS server 500 . The processor 501 corresponds to the central part of the computer. The processor 501 executes information processing for controlling each part to realize various functions of the POS server 500 in accordance with information processing programs such as an operating system and application programs.

[0026] The main memory unit 502 corresponds to the main memory portion of the computer. The main memory unit 502 includes a read-only memory area and a rewritable memory area. The main memory unit 502 stores a part of the information processing program in the read-only memory area. The main memory unit 502 may also store data required for the processor 501 to execute processes for controlling each part in the read-only memory area or the rewritable memory area. The main memory unit 502 uses the rewritable memory area as a work area for the processor 501.

[0027] The auxiliary storage unit 503 corresponds to the auxiliary storage part of the computer. The auxiliary storage unit 503 may be, for example, an EEPROM, a HDD, an SSD, or any other known storage device. The auxiliary storage unit 503 stores data used by the processor 501 in performing various processes and data generated by the processes in the processor 501. The auxiliary storage unit 503 may also store the information processing program. In this embodiment, the auxiliary storage unit 503 stores a dictionary management program PRB, which is one of the information processing programs. The dictionary management program PRB is an application program in which the procedure of the dictionary management process described later is described. A part of the storage area of ​​the auxiliary storage unit 503 is used as an area for storing dictionary data DAC. The dictionary data DAC is data representing an identification dictionary for identifying which product is reflected in an image. For example, when the transaction processing system 1 is introduced into a store, the dictionary data DAC is created by an appropriate worker in consideration of the products sold in the store and written into the auxiliary storage unit 503.

[0028] The communication unit 504 executes communication processing for performing data communication via the communication network 2. The communication unit 504 can use, for example, an existing communication device for the Internet. The transmission path 505 includes an address bus, a data bus, and control signal lines, and transmits data and control signals between the connected components.

[0029] Now, for example, a general-purpose server device can be used as the hardware of the POS server 500. The POS server 500 is generally transferred in a state in which the dictionary management program PRB is stored in the auxiliary storage unit 503. However, the hardware in a state in which the dictionary management program PRB is not stored in the auxiliary storage unit 503, or in a state in which a different version of the same type of application program is stored in the auxiliary storage unit 503, and the dictionary management program PRB may be transferred separately. The POS server 500 may be configured by writing the dictionary management program PRB to the auxiliary storage unit 503 in response to an operation by an arbitrary operator. The dictionary management program PRB may be transferred by recording it on a removable recording medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory, or by communication via a network.

[0030] Next, the operation of the transaction processing system 1 configured as above will be described. Note that the contents of the various processes described below are merely examples, and it is possible to change the order of some of the processes, omit some of the processes, or add other processes as appropriate. For example, in the following description, in order to easily explain the characteristic operations of this embodiment, the description of some of the processes is omitted. For example, when some error occurs, a process to deal with the error may be performed. Also, for example, in various standby states, the system may wait for the occurrence of an event different from that described, and may transition to a process according to the event that has occurred.

[0031] A customer of a sales store, that is, a purchaser who wishes to purchase an item displayed and sold at the sales store, performs a predetermined operation for starting use on an unused cart terminal 100. When this operation is performed, the processor 101 starts a registration process based on the cart terminal program PRA.

[0032] 3 and 4 are flowcharts of the registration process. 3, the processor 101 generates new transaction data DAB for the target transaction. That is, the processor 101 determines a new transaction code different from transaction codes for identifying other transactions according to a predetermined rule, generates new transaction data DAB that includes this transaction code but does not include product information on the purchased product, and stores the new transaction data DAB in the auxiliary storage unit 103.

[0033] In ACT12, the processor 101 checks whether it is time to capture an image. If the processor 101 cannot confirm the event, it determines NO and proceeds to ACT13. The capture timing is assumed to be, for example, immediately after proceeding from ACT11 to ACT12, and at regular time intervals thereafter. The capture timing may be appropriately determined, for example, by the creator of the cart terminal program PRA. However, the capture timing should be determined so that the image arrives at least once while the product to be put into the basket section 930 passes through the shooting range of the camera 199.

[0034] In ACT13, the processor 101 checks whether a payment has been requested. If the processor 101 cannot check the relevant event, it determines NO and returns to ACT12. Thus, in ACT12 and ACT13, the processor 101 waits for the acquisition timing to occur or for an accounting request to be made.

[0035] When the acquisition timing arrives, the processor 101 judges YES in ACT12 and proceeds to ACT14. As ACT14, the processor 101 acquires image data output by the camera 199. For example, if the camera 199 repeats photographing at a fixed cycle, the processor 101 acquires image data output by the camera 199 in response to the most recent photographing. For example, if the camera 199 photographs in response to an instruction from the processor 101, the processor 101 instructs the camera 199 to photograph, and acquires image data output by the camera 199 in response to the instruction. The processor 101 then stores the acquired image data in the main storage unit 102 or the auxiliary storage unit 103. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as an acquisition means for acquiring an image represented by the image data.

[0036] As ACT15, the processor 101 detects an area in which a product is reflected (hereinafter referred to as a product area) from the image represented by the acquired image data. The area to be extracted can be determined by using, for example, deep learning using a convolutional neural network (CNN) or a well-known method using graph cuts, and the method is not limited.

[0037] In ACT 16, the processor 101 checks whether a product area has been detected. If the processor 101 has not detected a product area in ACT 15, the processor 101 determines that the result is NO and proceeds to ACT 17. In ACT17, the processor 101 checks whether a candidate product (hereinafter, referred to as a candidate product) for registration as a purchased product has already been set. If the processor 101 cannot check the corresponding event, it judges NO and returns to the standby state of ACT12 and ACT13.

[0038] A purchaser searches for a product to be purchased in the store. Then, the purchaser takes out the product to be purchased from the sales floor and puts it in the basket section 930. At this time, the product passes through the shooting range of the camera 199 and is reflected in the image represented by the image data output by the camera 199. In other words, until the purchaser tries to put the product in the basket section 930, the processor 101 does not detect the product area in ACT15 even if the acquisition timing is reached, and no candidate products are set, so the processor 101 judges NO in ACT17 and repeats returning to the standby state of ACT12 and ACT13. Then, when the product that the purchaser is trying to put in the basket section 930 is reflected in the image, the processor 101 detects the product area in ACT15. In this case, the processor 101 judges YES in ACT16 and proceeds to ACT21 in FIG. 4.

[0039] The processor 101 executes an identification process as ACT21. The identification process is a process of identifying products reflected in a product area based on an image of the product area detected by ACT15 while referring to dictionary data DAA. The method of this identification process may be, for example, deep learning using CNN, which is a method suitable for image recognition. Alternatively, any method may be used, such as OCR (optical character recognition) or feature point matching. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as an identification means.

[0040] In ACT 22, the processor 101 checks whether the identification in ACT 21 was successful. If the identification was successful, the processor 101 judges YES and proceeds to ACT 23. In ACT23, the processor 101 checks whether a candidate product has already been set. If the processor 101 cannot check the event, it determines NO and proceeds to ACT24. In ACT 24, the processor 101 sets the product identified in ACT 21 as a candidate product. Then, the processor 101 returns to the standby state of ACT 12 and ACT 13 in FIG.

[0041] There may be cases where multiple acquisition timings occur while one purchased product passes through the shooting range of camera 199. In this case, processor 101 identifies the same product by the identification process in ACT21 at each of the multiple acquisition timings, but candidate products have already been set at the second and subsequent acquisition timings. Therefore, processor 101 judges YES in ACT23 in Fig. 4, skips ACT24, and returns to the standby state of ACT12 and ACT13 in Fig. 3.

[0042] Thus, while the purchaser is attempting to place the product he or she wishes to purchase into the basket section 930 and the product passes through the shooting range of the camera 199, the processor 101 repeats ACT14 to ACT16 in Figure 3 and ACT21 to ACT23 in Figure 4 each time an acquisition timing occurs, and during this time, the product that was first identified in ACT21 remains the candidate product.

[0043] Then, when the product that the purchaser is about to put in the basket section 930 passes through the shooting range of the camera 199 and is no longer reflected in the image obtained by the camera 199, the processor 101 is no longer able to detect the product area in ACT15 in Fig. 3. In response to this, the processor 101 judges NO in ACT16 and proceeds to ACT17, but because the candidate products have already been set as described above, the processor 101 judges YES in ACT17 and proceeds to ACT18.

[0044] In ACT18, the processor 101 checks whether a predetermined determination condition for determining the candidate product as the purchased product is satisfied. The determination condition may be appropriately determined, for example, by the creator of the cart terminal program PRA. As an example, the determination condition is assumed to be determined as "YES is determined in ACT17 at each of the specified number of consecutive acquisition timings." If the determination condition is not satisfied, the processor 101 determines NO and returns to the standby state of ACT12 and ACT13. Thus, even if the candidate product has been set and the product area cannot be detected, the processor 101 repeatedly executes the processes of ACT14 to ACT18 at each acquisition timing until the determination condition is satisfied. If the determination condition is satisfied, the processor 101 determines YES in ACT18 and proceeds to ACT19.

[0045] In ACT19, the processor 101 registers the candidate product as a purchased product. That is, the processor 101 updates the transaction data to include the product code of the candidate product as the product code of the purchased product, for example. Then, the processor 101 returns to the standby state of ACT12 and ACT13. The processor 101 cancels the setting of a product that has been registered as a purchased product as a candidate product. Therefore, until the processor 101 can detect a new product area in ACT15, the processor 101 judges NO in ACT16 and proceeds to ACT17. However, since the processor 101 cannot confirm the existence of a candidate product in ACT17, the processor 101 judges NO and returns to the standby state of ACT12 and ACT13.

[0046] Now, when the dictionary data DAA is no longer appropriate due to a change in the packaging of the product reflected in the product area, or when the dictionary data DAA does not correspond to the product reflected in the product area, the processor 101 may not be able to identify the product in ACT21 in Fig. 4. Alternatively, even if the dictionary data DAA is appropriate, the product may not be identified in ACT21 if the product is covered by a hand, for example, and the product's features are not sufficiently visible in the image for identification. In such a case, the processor 101 judges NO in ACT22 and proceeds to ACT25.

[0047] In ACT25, the processor 101 checks whether the recognition failure (hereinafter referred to as recognition error) is due to a defect in the dictionary data DAA. For example, the processor 101 checks whether a predetermined condition for determining that there is a recognition error is met. The condition may be determined as appropriate by the creator of the cart terminal program PRA, but as an example, it is assumed that the condition is determined as "NO was determined in ACT22 at each of the specified number of consecutive acquisition timings." If the processor 101 cannot confirm that there is a recognition error, it determines NO and returns to the standby state of ACT12 and ACT13 in FIG. 3. If a rule is set that when placing a purchased item in the basket 930, the item must be held so that it can be clearly photographed by the camera 199, then even if a temporary situation occurs in which the item is covered by a hand and recognition fails, the situation will be resolved within a short time. Therefore, the above conditions make it possible to accurately determine whether or not a recognition error has occurred.

[0048] If the processor 101 can confirm that an identification error has occurred, for example, because a certain condition is met, the processor 101 judges YES in ACT25 in FIG. 4 and proceeds to ACT26. As ACT26, the processor 101 executes a guidance operation. The guidance operation is an operation for informing the purchaser to hold the barcode displayed on the product to be put in the basket section 930 over the camera 199. As the guidance operation, for example, the processor 101 causes the touch panel 104 to display a guidance screen showing a text message for making the purchaser recognize that the barcode displayed on the product to be put in the basket section 930 should be held over the camera 199 and an image output by the camera 199. As the guidance operation, for example, the processor 101 causes the sound unit 105 to output an alarm sound for making the purchaser recognize the occurrence of an abnormality. As the guidance operation, for example, the processor 101 causes the sound unit 105 to output a voice message for making the purchaser recognize that the barcode displayed on the product to be put in the basket section 930 should be held over the camera 199. In addition, the operation performed as the guidance operation may be appropriately determined by, for example, the creator of the cart terminal program PRA, and may be performed by combining a plurality of the above-mentioned operations. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as a guide means.

[0049] As ACT27, the processor 101 executes a barcode recognition process. For example, the processor 101 detects a barcode captured in the latest image captured by the camera 199, and attempts to extract a product code from the data represented by the barcode. If the product code is successfully extracted, the processor 101 acquires the product code as an identifier of the product. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as a reading means for reading a product code as an identifier of the product from a barcode, which is an example of a visible code.

[0050] In ACT28, the processor 101 checks whether the product code has been extracted by the barcode recognition process. If the product code has not been extracted, the processor 101 determines NO and returns to ACT27. Note that when the processor 101 executes ACT27 for the second or subsequent time after ACT28, it acquires an image different from the image acquired previously and processes that image.

[0051] If the barcode displayed on the product is held up to the camera 199 by the purchaser, causing the barcode to be clearly captured in the image obtained by the camera 199, and the product code can be extracted in ACT27, the processor 101 judges YES in ACT28 and proceeds to ACT29. In ACT29, the processor 101 registers, as a purchased product, the product identified by the product code extracted in ACT27. That is, the processor 101 updates the transaction data so that the product code extracted in ACT27 is included as the product code of the purchased product, for example.

[0052] In ACT 30, the processor 101 transmits error data. The processor 101 generates error data as data including at least the image data acquired in ACT 14 and the product code extracted in ACT 27. In this way, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as the central part functions as a generating means. As ACT31, the processor 101 transmits the error data generated in ACT30 from the wireless communication unit 107 to the communication network 2, addressed to the POS server 500. The processor 101 then returns to the standby state of ACT12 and ACT13 in Fig. 3. Such transmission of error data is an example of outputting error data. In other words, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as an output means. Incidentally, even if the processor 101 fails to identify in ACT21 in FIG. 4, if it cannot confirm that there is an identification error, it judges NO in ACT25, passes ACT26 to ACT30, and returns to the standby state in ACT12 and ACT13 in FIG.

[0053] Once the purchaser has finished registering the purchased items, the purchaser performs a predetermined operation, for example on touch panel 104, to indicate the start of a transaction. In response, processor 101 determines YES in ACT13 in FIG. 3 and proceeds to the transaction process. The transaction process may be the same as that performed in existing cart POS systems, and is not shown in the figures. For example, processor 101 performs the transaction process by transferring transaction data to transaction device 200. Alternatively, if a payment terminal (not shown) is attached to cart 900 and the payment terminal is connected to cart terminal 100, the transaction process may be the process by which credit card payment or code payment is performed using the payment terminal.

[0054] In the POS server 500, a processor 501 executes a dictionary management process based on a dictionary management program PRB. FIG. 5 is a flowchart of the dictionary management process. In ACT 41, the processor 501 checks whether error data has been received. If the processor 501 cannot check the relevant event, it determines NO and proceeds to ACT 42.

[0055] In ACT 42, the processor 501 checks whether the learning timing has arrived. If the processor 501 cannot check the relevant event, it determines NO and returns to ACT 41. Thus, the processor 501 as ACT41 and ACT42 waits for the receipt of error data or the arrival of a learning timing.

[0056] As described above, when the error data sent from the cart terminal 100 is transmitted to the POS server 500 via the communication network 2, the error data is received by the communication unit 504. Then, the processor 501 judges YES in ACT41 and proceeds to ACT43. As ACT43, the processor 501 stores the error data received by the communication unit 504 in the auxiliary storage unit 503. Then, the processor 501 thereafter returns to a standby state for ACT41 and ACT42.

[0057] Thus, the processor 501 repeats ACT43 every time error data is received by the communication unit 504. At this time, the processor 501 additionally stores the newly received error data in the auxiliary storage unit 503 without deleting the error data already stored in the auxiliary storage unit 503. In other words, when error data is repeatedly sent, the processor 501 accumulates the error data in the auxiliary storage unit 503. Thus, the processor 501 executes information processing based on the dictionary management program PRB, and the computer with the processor 501 as its central part functions as a collection means.

[0058] The learning timing may be appropriately determined, for example, by the creator of the dictionary management program PRB or the administrator of the transaction processing system 1. As one example, the learning timing is expected to be determined as the timing after the closing of business on one business day of the store where the transaction processing system 1 is used. When the learning timing arrives, the processor 501 judges YES in ACT42 and proceeds to ACT44.

[0059] In ACT44, the processor 501 checks whether there is error data. If the error data is not stored in the auxiliary memory unit 503, the processor 501 judges NO and returns to the standby state of ACT41 and ACT42. However, if the error data is stored in the auxiliary memory unit 503, the processor 501 judges YES in ACT44 and proceeds to ACT45.

[0060] As ACT45, processor 501 checks whether dictionary learning based on error data is set automatically. Note that this setting is set by processor 501 in response to an instruction by an administrator of transaction processing system 1, for example, operating management terminal 400, as an operation setting related to dictionary management processing.

[0061] If the setting is not automatic, the processor 501 determines NO and proceeds to ACT46. As ACT46, the processor 501 executes a notification operation. The notification operation is an operation for notifying the administrator of the transaction processing system 1 that a product has been registered by reading a barcode in response to an event in which error data is generated, i.e., an identification error has occurred. The processor 501, as a guide operation, requests the management terminal 400 to display a notification screen showing a text message or the like for the administrator to recognize that the dictionary data DAC needs to be updated. In response to such a request, the management terminal 400 performs an operation such as displaying the corresponding notification screen. In this embodiment, the notification screen is a screen that also serves as a GUI screen for receiving an instruction from the administrator as to whether or not to utilize the error data for dictionary learning. When the administrator receives a notification from such a notification operation, the administrator can recognize that the dictionary data DAC needs to be updated, so the notification operation can also be said to be an operation that recommends the administrator to update the dictionary data DAC. Thus, the processor 501 executes information processing based on the dictionary management program PRB, and the computer with the processor 501 as its central part functions as a recommendation means.

[0062] In ACT47, the processor 501 checks whether the use of error data has been instructed. If an operation to instruct the use is performed by the administrator on the management terminal 400, for example, and this is notified from the management terminal 400 to the POS server 500, the processor 501 judges the result as YES and proceeds to ACT48. If the dictionary learning is set to be performed automatically, the processor 501 judges the result as YES in ACT45, skips ACT46 and ACT47, and proceeds to ACT48.

[0063] As ACT48, the processor 501 performs a learning process. That is, the processor 501 updates the dictionary data DAC using the error data. That is, the processor 501 extracts the external characteristics of the product reflected in the image represented by the image data included in the error data, and adds data to the dictionary data DAC for use in the recognition process as the characteristics of the product identified by the product code included in the same error data. Note that the specific process of updating the dictionary data DAC here may be, for example, a well-known process for updating the same type of dictionary data. Note that the dictionary data DAC is downloaded to the cart terminal 100 at the subsequent dictionary download timing and rewritten as the dictionary data DAA, so that the update of the dictionary data DAC is also reflected in the dictionary data DAA. Thus, the processor 501 executes information processing based on the dictionary management program PRB, and the computer with the processor 501 as the central part functions as an update means. In ACT49, the processor 501 erases the error data used in the learning process in ACT48 from the auxiliary storage unit 503. Then, the processor 501 thereafter returns to the standby state for ACT41 and ACT42.

[0064] When an instruction not to utilize the error data is given, the processor 501 judges NO in ACT47, passes ACT48 and ACT49, and returns to the standby state of ACT41 and ACT42. In this case, the administrator should study the dictionary by a method other than the dictionary management process. Therefore, when the processor 501 judges NO in ACT47, the processor 501 may perform an operation to support the updating of the dictionary data DAC, such as requesting the management terminal 400 to display a screen showing a list of products identified by the product codes included in the error data, that is, products that are difficult to recognize based on the dictionary data DAC.

[0065] As described above, when the transaction processing system 1 fails to identify a product based on the external characteristics of the product reflected in the image and the product is registered by reading the barcode, it generates error data including the image data that was the subject of the failed identification process and the product code read from the barcode. This enables the administrator of the transaction processing system 1 to recognize the possibility that the dictionary data DAA, DAC may be inappropriate based on the error data. Note that the image data and product code included in the error data are both obtained in response to the purchaser's operation to register the purchased product, so no special operation is required to generate the error data.

[0066] Furthermore, transaction processing system 1 transmits error data generated by cart terminal 100 to POS server 500, which collects the data. This allows error data generated by each of multiple cart terminals 100 to be centrally managed by POS server 500. An administrator can check the error data by accessing POS server 500 from management terminal 400, for example, without operating each of the multiple cart terminals 100.

[0067] Furthermore, if the transaction processing system 1 is unable to identify the product captured in the image by the identification process, the transaction processing system 1 guides the purchaser to perform an operation for reading the barcode by a guidance operation, thereby making it possible to obtain the product code to be included in the error data more reliably.

[0068] Furthermore, in response to the generation of error data, transaction processing system 1 executes a notification operation, thereby making the administrator aware that the dictionary data should be updated.

[0069] Furthermore, the transaction processing system 1 updates the dictionary data based on the error data, which eliminates the need for the administrator to perform a separate operation to input data for updating the dictionary.

[0070] Furthermore, if the transaction processing system 1 is set to automatically perform dictionary learning based on error data, the transaction processing system 1 updates the dictionary data based on the error data without performing a notification operation. This means that the administrator does not need to perform any operation to update the dictionary data based on error data.

[0071] This embodiment can be modified in various ways as follows. The device may be used in combination with another product registration device or transaction processing device, and may be realized as a product identification device whose main function is to identify products.

[0072] The results of identifying the product may be used for any purpose other than purchase registration for transaction processing.

[0073] In the above embodiment, a part of the processing executed by the processor 101 may be executed by the POS server 500 or any other one or more information processing devices. For example, the cart terminal 100 may perform only the user interface operation, and various information processing for identifying the purchased item may be executed by a computer provided in the POS server 500. In other words, each of the acquiring means, the identifying means, the reading means, and the generating means may be provided in any information processing device such as the POS server 500 other than the cart terminal 100 included in the transaction processing system 1, or may be appropriately distributed among multiple information processing devices included in the transaction processing system 1.

[0074] The processor 101 in the cart terminal 100 may update the dictionary data DAA stored in the auxiliary storage unit 103 based on the error data.

[0075] Each function realized by the processor 101 through information processing can be realized in part or in whole by hardware that executes information processing not based on a program, such as a logic circuit, etc. Also, each of the above functions can be realized by combining the above hardware, such as the logic circuit, with software control.

[0076] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0077] 1...transaction processing system, 2...communication network, 100...cart terminal, 101...processor, 102...main memory unit, 103...auxiliary memory unit, 104...touch panel, 105...sound unit, 106...interface unit, 107...wireless communication unit, 108...transmission path, 199...camera, 200...accounting machine, 300...attendant terminal, 400...management terminal, 500...POS server, 501...processor, 502...main memory unit, 503...auxiliary memory unit, 504...communication unit, 505...transmission path, 900...cart.

Claims

1. An acquisition means for acquiring an image of a product that is to be placed in a container by a purchaser for the purpose of purchase; A discrimination means for discriminating a product based on the characteristics of the product shown in the image acquired by the acquisition means and dictionary data representing the characteristics of the appearance of each of a plurality of products; A reading means for reading an identifier of the product from a visible code displayed on the product; a generating means for generating error data for managing an identification error when a product is captured in an image acquired by the acquiring means but cannot be identified by the identifying means and an identifier is read by the reading means for the product that could not be identified, the error data including the image acquired by the acquiring means and the identifier read by the reading means; A product identification system comprising:

2. The present invention comprises a product identification device and a dictionary management device, The product identification device the acquiring means, the identifying means, the reading means, and the generating means; an output unit that outputs the error data generated by the generation unit to the dictionary management device, The dictionary management device includes: a collection means for collecting the error data output by the output means; Equipped with The product identification system of claim 1 .

3. The product identification device a guidance means for guiding a purchaser to perform an operation for causing said reading means to read a visible code displayed on the product when the product is reflected in the image acquired by said acquisition means but cannot be identified by said identification means; The product identification system of claim 2 further comprising:

4. The dictionary management device includes: a recommendation means for recommending to an administrator that the dictionary data be updated in response to the generation of error data by the generation means; The product identification system of claim 2 further comprising:

5. The dictionary management device includes: an update means for updating the dictionary data based on the error data generated by the generation means; The product identification system of claim 2 further comprising:

Citation Information

Patent Citations

  • Dictionary enhancement support system, dictionary enhancement support method, and dictionary enhancement support program

    JP2005267578A

  • Fraud prevention system and fraud prevention program

    JP2021135620A