Item recognition system, information processing program, and transaction processing system

Abstract

To prevent a wrong recognition result from being used.SOLUTION: The item recognition system according to an embodiment includes identification means, recognition means, detection means, reception means, and confirmation means. The identification means identifies a user. The recognition means recognizes an item represented in an image on the basis of characteristics of the item. The detection means detects a low possibility that the identified person selects the recognized item. The reception means receives the specification that the recognized item is an item that the identified user selected, in response to detection that the possibility of the selection is low made by the selection means. The confirmation means confirms that the recognized item is an item that the identified user selected, in a case where that a low possibility of the selection is not detected and in a case where the specification that the recognized item is an item that the identified user selected is received.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] An embodiment of the present invention relates to an item recognition system, an information processing program, and a transaction processing system. [Background technology]

[0002] A registration system has already been proposed for display sales stores that automatically registers the products that customers will purchase by recognizing the images of the products that they put in their shopping carts. POS systems that use such registration systems are sometimes called frictionless POS systems. However, there are cases where the appearances of multiple separate products are similar, and it is difficult to completely avoid misidentification based on the characteristics shown in the image. For these reasons, measures to address such misconceptions were needed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-150436 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide an article recognition system, an information processing program, and a transaction processing system that can prevent erroneous recognition results from being used. [Means for solving the problem]

[0005] An embodiment of an item recognition system includes an identification means, a recognition means, a detection means, a reception means, and a determination means. The identification means identifies a user. The recognition means recognizes an item based on features of the item appearing in an image. The detection means detects that the user identified by the identification means is unlikely to select the item recognized by the recognition means. In response to the detection means detecting the unlikely selection, the reception means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means. The determination means determines the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect the unlikely selection and when the reception means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system according to an embodiment and a main circuit configuration of a cart terminal. [Diagram 2] FIG. 13 is a perspective view showing an example of a state in which the cart terminal is attached to the cart. [Diagram 3] 13 is a flowchart of a registration process. [Figure 4] 13 is a flowchart of a registration process. [Diagram 5] FIG. 13 is a diagram illustrating an example of a registration screen. [Figure 6] FIG. 13 is a diagram illustrating an example of a confirmation screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] An example of an embodiment will be described below with reference to the drawings. Note that as this embodiment, an example of a transaction processing system implemented by incorporating the functions of an item recognition system will be described. FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system 1 according to this embodiment and a main circuit configuration of a cart terminal 100. As shown in FIG.

[0008] The transaction processing system 1 is an information processing system for processing sales of products to customers in a store. The transaction processing system 1 is configured so that a cart terminal 100, a billing machine 200, an attendant terminal 300, a management terminal 400, and a POS server 500 can communicate with each other via a communication network 2.

[0009] The communication network 2 may be the Internet, a virtual private network (VPN), a local area network (LAN), a public communication network, a mobile communication network, or the like, either alone or in appropriate combination. As an example, the communication network 2 may be a combination of a LAN and the Internet. Although any number of cart terminals 100, accounting machines 200, attendant terminals 300, control terminals 400, and POS servers 500 may be included in transaction processing system 1, only one of each is shown in FIG.

[0010] The cart terminal 100 is an information processing terminal attached to a shopping cart (hereinafter referred to as a cart) provided in a store. The cart terminal 100 is loaned together with the cart to a customer who purchases products at the store. The cart terminal 100 is a terminal device that executes a registration process for registering a product that the customer plans to purchase as a transaction product in response to an operation by the customer. As will be described later, the cart terminal 100 executes a process for recognizing a product to be registered as a transaction product from an image in which the product is reflected. In other words, the cart terminal 100 has a function as a product recognition device. A product is an example of an article.

[0011] The payment machine 200 is installed in a store, and executes payment processing related to the calculation and settlement of the price of the transaction goods registered by the cart terminal 100. The payment machine 200 is operated by an operator during payment processing. The operator of the payment machine 200 is usually a customer. In some cases, the operator of the payment machine 200 is a store clerk. As the payment machine 200, for example, one that is used in an existing cart POS system can be used as is.

[0012] The attendant terminal 300 is an information processing terminal operated by a store clerk. The attendant terminal 300 is a terminal device for a user interface related to information processing to support the work of the store clerk regarding the transaction processed by the transaction processing system 1. The work of the store clerk is, for example, monitoring the implementation status of the transaction being processed and supporting the customer appropriately. As the basic hardware of the attendant terminal 300, various general-purpose information processing devices such as desktop type, notebook type, tablet type, and mobile type can be used. The attendant terminal 300 can be realized by installing an information processing program for realizing various functions as the attendant terminal 300 in a computer equipped with such basic hardware.

[0013] Management terminal 400 is an information terminal that serves as a user interface for an administrator who manages transaction processing system 1. As basic hardware for management terminal 400, various general-purpose information processing devices such as desktop type, notebook type, tablet type, and mobile type can be used. Management terminal 400 can be realized by installing an information processing program for realizing various functions of management terminal 400 in a computer equipped with such basic hardware.

[0014] The POS server 500 is an information processing device that executes management processing for managing merchandise buying and selling transactions that are processed by the registration processing in the cart terminal 100 and the accounting processing in the accounting device 200. As the basic hardware of the POS server 500, an appropriate information processing device such as a general-purpose server device can be used. The POS server 500 can be realized by installing an information processing program for realizing various functions of the POS server 500 in a computer equipped with such basic hardware.

[0015] The cart terminal 100 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, a touch panel 104, a sound unit 105, an interface unit 106, a wireless communication unit 107, and a transmission path 108. The processor 101 is capable of communicating with the main memory unit 102, the auxiliary memory unit 103, the touch panel 104, the sound unit 105, the interface unit 106, and the wireless communication unit 107 via the transmission path 108.

[0016] The processor 101, the main storage unit 102, and the auxiliary storage unit 103 are connected via a transmission line 108 to constitute a computer that performs information processing for controlling the cart terminal 100. The processor 101 corresponds to the central part of the computer. The processor 101 executes information processing for controlling each part to realize various functions of the cart terminal 100 in accordance with information processing programs such as an operating system and application programs.

[0017] The main memory unit 102 corresponds to the main memory portion of the computer. The main memory unit 102 includes a read-only memory area and a rewritable memory area. The main memory unit 102 stores a part of the information processing program in the read-only memory area. The main memory unit 102 may also store data required for the processor 101 to execute processes for controlling each part in the read-only memory area or the rewritable memory area. The main memory unit 102 uses the rewritable memory area as a work area for the processor 101.

[0018] The auxiliary storage unit 103 corresponds to the auxiliary storage portion of the computer. The auxiliary storage unit 103 may be, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), an SSD (solid state drive), or any other known storage device. The auxiliary storage unit 103 stores data used by the processor 101 in performing various processes and data generated by the processes in the processor 101. The auxiliary storage unit 103 may also store the information processing program. In this embodiment, the auxiliary storage unit 103 stores a cart terminal program PRA, which is one of the information processing programs. The cart terminal program PRA is an application program in which the procedure of the registration process is described. A part of the storage area of ​​the auxiliary storage unit 103 is used as an area for storing transaction data DAA. The transaction data DAA is data representing the contents of one transaction.

[0019] The touch panel 104 displays a screen for presenting information to the operator, and also inputs instructions via the touch operation on the screen by the operator. The sound unit 105 outputs sounds for various guidance and warnings. As the sound unit 105, various well-known sound devices such as a voice synthesis device and a buzzer can be used alone or in combination.

[0020] An external device such as a camera 199 is connected to the interface unit 106. The interface unit 106 interfaces for sending and receiving data to and from the connected external device. An existing USB (universal serial bus) controller or the like can be used as the interface unit 106. The camera 199 may repeat photographing at regular intervals, or may photograph in response to an instruction from the processor 101. A camera built into the cart terminal 100 may be used instead of the camera 199.

[0021] The wireless communication unit 107 executes communication processing for wirelessly performing data communication via the communication network 2. For example, an existing wireless communication device for wireless LAN can be used as the wireless communication unit 107. Note that instead of or in addition to the wireless communication unit 107, a communication unit connected to the communication network 2 by wire may be used. The transmission path 108 includes an address bus, a data bus, and control signal lines, and transmits data and control signals between the connected components.

[0022] The basic hardware of the cart terminal 100 is assumed to be, for example, the hardware of a tablet-type information processing device. The cart terminal 100 is generally transferred in a state in which the cart terminal program PRA is stored in the auxiliary storage unit 103. However, the cart terminal program PRA may be transferred separately from the hardware in a state in which the cart terminal program PRA is not stored in the auxiliary storage unit 103, or in a state in which a different version of the same type of application program is stored in the auxiliary storage unit 103. The cart terminal 100 may be configured by writing the cart terminal program PRA to the auxiliary storage unit 103 in response to an operation by an arbitrary worker. The cart terminal program PRA may be transferred by recording it on a removable recording medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory, or by communication via a network.

[0023] FIG. 2 is a perspective view showing an example of how the cart terminal 100 is attached to the cart. The cart 900 includes a caster portion 910 , a handle frame portion 920 , a basket portion 930 , a support portion 940 , and a battery case 950 . The caster unit 910 has four wheels 911 for smoothly moving the cart 900 on a floor surface. The wheels 911 are attached to a frame 912 in a state capable of rotating around an axis in the vertical direction.

[0024] The handle frame portion 920 includes a pair of vertical frames 921, 921 and a handle bar 922. The vertical frames 921, 921 are erected above the two wheels of the caster portion 910. The handle bar 922 connects the upper ends of the vertical frames 921, 921 together. The basket section 930 is provided horizontally from a midway portion of the handle frame section 920. The basket section 930 is shaped like a basket with an opening at the top to allow products to be placed therein, and functions as a container for storing products that a customer wishes to purchase. A camera 199 is attached to the upper end of the basket section 930 on the side closer to the handle bar 922 in a state in which the camera 199 can photograph the opening of the basket section 930 from the side. The photographing range of the camera 199 is determined so that the products placed inside the basket section 930 are captured through the opening of the basket section 930.

[0025] The support unit 940 includes a pole 941. The pole 941 is attached to one of the vertical frames 921 so that its tip is positioned higher than the handlebars 922. The cart terminal 100 described above is attached to the tip of the pole 941. As a result, the support unit 940 supports the cart terminal 100 in the state shown in FIG. The battery case 950 is attached between the vertical frames 921, 921 on the lower end side of the handle frame portion 920. The battery case 950 houses a battery used as an external power source for the cart terminal 100.

[0026] Next, the operation of the transaction processing system 1 configured as above will be described. Note that the contents of the various processes described below are merely examples, and it is possible to change the order of some of the processes, omit some of the processes, or add other processes as appropriate. For example, in the following description, in order to easily explain the characteristic operations of this embodiment, the description of some of the processes is omitted. For example, when some error occurs, a process to deal with the error may be performed. Also, for example, in various standby states, the system may wait for the occurrence of an event different from that described, and may transition to a process according to the event that has occurred.

[0027] A customer of a retail store, that is, a customer who wishes to purchase a product displayed and sold at the retail store, performs a predetermined operation for starting use on an unused cart terminal 100. When this operation is performed, the processor 101 starts a registration process based on the cart terminal program PRA.

[0028] 3 and 4 are flowcharts of the registration process. The processor 101 performs a login process as ACT11 in FIG. 3. For example, the processor 101 displays a login screen on the touch panel 104 for the customer to specify a user code and password, and receives the customer's specification of the user code and password. The processor 101 then authenticates whether the customer is a registered user, for example, based on the user code and password specified by the customer, using a well-known user authentication technique. If the authentication is successful, the processor 101 stores the specified user code as the customer's identification code, for example, in the main storage unit 102 or the auxiliary storage unit 103. Note that the cart terminal 100 may be provided with a reader that reads the user code recorded on the user card, and the processor 101 may perform user authentication using the user code when the user code is read by the reader, or may store the user code as the customer's identification code without performing user authentication. The processor 101 also allows login without obtaining a user code. In other words, the processor 101 may receive a designation for visitor use on the login screen, and in response to that designation, without acquiring a user code, stores a predetermined visitor code as a customer identification code, for example, in the main memory unit 102 or the auxiliary memory unit 103.

[0029] In ACT12, the processor 101 generates new transaction data DAA for the target transaction. That is, the processor 101, for example, determines a new transaction code different from the transaction code serving as an identifier for identifying other transactions in accordance with a predetermined rule, generates new transaction data DAA including this transaction code but not including product information related to the transaction product, and stores the new transaction data DAA in the auxiliary storage unit 103. As ACT13, the processor 101 displays a registration screen on the touch panel 104. The registration screen is a screen for allowing a customer to recognize the registration status of a transaction product.

[0030] FIG. 5 is a diagram showing an example of the registration screen. In addition, in the screen examples shown in FIG. 5 and other figures, some display objects may be omitted from the illustration. In the registration screen shown in FIG. 5, an image showing the registration status is placed in the display area ARA. The registration screen shows the button BUA. In the example of FIG. 5, the image placed in the display area ARA shows that a total of two items have been registered, one each of the items with the product names "AAAAAAAAAA" and "BBBBBBBBBB", with unit prices of 178 yen and 198 yen, respectively, and the reference price is 406 yen. The reference price is the amount obtained by subtracting the discount amount of various services such as coupon services from the total unit price of all the transaction items. If the transaction is settled without changing the transaction items and the applied services, this reference price will be the settlement amount. The image placed in the display area ARA changes sequentially according to the registration status of the transaction items. However, since the transaction items have not yet been registered when ACT13 is executed, the display area ARA at this time does not show the information of each item, and an image showing a total of 0 items and a reference price of 0 yen is placed. The button BUA is a soft key for the operator to instruct the transaction. The contents of the registration screen may be determined as appropriate by, for example, the creator of the cart terminal program PRA.

[0031] In ACT 14, the processor 101 checks whether it is time to capture an image. If the processor 101 cannot check the event, it determines NO and proceeds to ACT 15. In ACT 15, the processor 101 checks whether a payment has been requested. If the processor 101 cannot check the relevant event, it determines NO and returns to ACT 14. Thus, in ACT14 and ACT15, the processor 101 waits for the acquisition timing to occur or for an account to be specified.

[0032] The acquisition timing is assumed to be, for example, the timing immediately after proceeding from ACT13 to ACT14, and the timing at regular time intervals thereafter. The acquisition timing may be appropriately determined, for example, by the creator of the cart terminal program PRA. However, the acquisition timing should be determined so that the signal arrives at least once, and preferably multiple times, while the commodity to be put into the basket section 930 passes through the shooting range of the camera 199.

[0033] When the acquisition timing arrives, the processor 101 determines YES in ACT14 and proceeds to ACT16. In ACT16, processor 101 acquires image data output by camera 199. For example, if camera 199 repeats shooting at a constant cycle, processor 101 acquires image data output by camera 199 in response to the first shooting after the acquisition timing. For example, if camera 199 shoots in response to an instruction from processor 101, processor 101 instructs camera 199 to shoot, and acquires image data output by camera 199 in response to the instruction. Then, processor 101 stores the acquired image data in main storage unit 102 or auxiliary storage unit 103.

[0034] As ACT17, the processor 101 detects an area in which a product is reflected (hereinafter referred to as a product area) from the image represented by the acquired image data. The area to be detected can be determined by using, for example, deep learning using a convolutional neural network (CNN) or a well-known method using graph cuts, and the method is not limited.

[0035] In ACT 18, the processor 101 checks whether the detection of the product area was successful. If the processor 101 could not detect the product area in ACT 17, the result is NO, and the processor 101 proceeds to ACT 19. Thus, the processor 101 proceeds to ACT 19 when the product is not captured in the image captured by the camera 199.

[0036] In ACT19, the processor 101 checks whether the product is being recognized. If the relevant event cannot be confirmed as described below, the processor 101 judges NO and returns to the standby state of ACT14 and ACT15. The customer searches for the product to be traded in the store. The customer then takes the product to be purchased from the sales floor and places it in the basket section 930. At this time, the product enters the shooting range of the camera 199 and is reflected in the image represented by the image data output by the camera 199. In other words, until the customer tries to put the product in the basket section 930, the processor 101 does not detect the product area in ACT17 even if the acquisition timing arrives, and product recognition has not started. Therefore, the processor 101 judges NO in both ACT18 and ACT19 and returns to the standby state of ACT14 and ACT15, and repeats this process. When the product that the customer is trying to put in the basket section 930 is reflected in the image, the processor 101 detects the product area in ACT17. In this case, the processor 101 determines YES in ACT18 and proceeds to ACT20.

[0037] In ACT20, the processor 101 checks whether the product is being recognized. If the processor 101 cannot check the relevant event as described below, it determines NO and proceeds to ACT21. In ACT21, the processor 101 performs a recognition start process. The recognition start process is a process for making it possible to confirm that a commodity that has entered the shooting range of the camera 199 is being recognized. For example, the processor 101 sets a recognition in progress flag to a state indicating that recognition is in progress.

[0038] As ACT22, the processor 101 executes a recognition process. The recognition process is a process for recognizing a product reflected in a product area based on an image of the product area detected in ACT17. The recognition process may be either a process for product recognition based on the appearance of the product or a process for product recognition based on a product barcode displayed on the product, or both of them. For the process for product recognition based on the appearance of the product, it is assumed that deep learning using CNN, which is a method suitable for image recognition, is used. Alternatively, for the process for product recognition based on the appearance of the product, any method may be used, such as OCR (optical character recognition) or feature point matching. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as the central part functions as a recognition means. Furthermore, if a process for product recognition based on a product barcode displayed on the product is applied to the recognition process, another device such as a laser scanner that detects a pattern of a product barcode without being based on an image may be used instead of or in addition to the camera 199.

[0039] In ACT23, the processor 101 checks whether the recognition in ACT21 was successful. If the recognition process does not result in successful recognition of the product, the processor 101 determines NO in ACT23 and returns to the standby state of ACT14 and ACT15. In other words, even if the product is captured in the image, if the processor 101 is unable to recognize the product from the image, the processor 101 returns to the standby state of ACT14 and ACT15.

[0040] If the processor 101 proceeds to ACT20 again because the same product remains in the image at the subsequent acquisition timing, the processor 101 judges the result to be YES, for example, if the recognition in progress flag indicates that the product is being recognized, and proceeds to ACT22, skipping ACT21.

[0041] Now, if the recognition in ACT 22 is successful, the processor 101 judges YES in ACT 23 and proceeds to ACT 31 in FIG. In ACT 31, the processor 101 checks whether the membership code has been acquired. If the membership code acquired in ACT 11 is stored in the main memory unit 102 or the auxiliary memory unit 103, the processor 101 judges that the membership code has been acquired (YES) and proceeds to ACT 32. Note that when the processor 101 acquires the membership code in ACT 11, the processor 101 may set an acquisition flag to a state indicating that the membership code has been acquired, and make a judgment based on the acquisition flag in ACT 31.

[0042] As ACT32, the processor 101 executes a possibility determination process. The possibility determination process is a process for determining the possibility that a customer identified by the acquired membership code will purchase the product recognized in ACT21 in FIG. 3 (hereinafter referred to as the recognized product). As an example, the processor 101 refers to information such as attribute information or purchase history of the customer identified by the membership code to obtain a numerical value representing the possibility that the customer will purchase the recognized product. More specifically, for example, the processor 101 determines the product category to which the recognized product belongs, obtains the customer's purchase frequency for all products belonging to the product category based on the purchase history, and sets this purchase frequency as a numerical value representing the possibility. That is, as an example, it is assumed that the recognized product belongs to a product category called "cosmetics for women". In this case, the processor 101 obtains the purchase frequency for all products belonging to the product category called "cosmetics for women" from the purchase history of the customer identified by the acquired membership code, and sets this numerical value as a numerical value representing the possibility. Also, as an example, when the recognized product belongs to the product category of "cosmetics for women" and the gender included in the customer's attribute information is "male," the processor 101 lowers the numerical value representing the possibility.

[0043] The processor 101 may determine the purchase history from purchase history data managed in a well-known manner, for example, by the POS server 500 or a server separately provided for managing the purchase history. The processor 101 may use attribute information managed in a well-known manner, for example, by the POS server 500 or a server separately provided for member management, for the customer attribute information.

[0044] In ACT33, the processor 101 checks whether the possibility of purchase is low based on the numerical value representing the possibility determined as described above. For example, the processor 101 determines that there is a possibility of purchase when the numerical value obtained in ACT33 satisfies a predetermined condition. The condition here is assumed to be determined as "the numerical value obtained in ACT33 is equal to or less than a specified value". However, the above condition may be "the numerical value obtained in ACT33 is less than a preset value". Alternatively, in ACT33, a numerical value is determined so that the higher the possibility, the smaller the numerical value becomes, and the above condition may be "the numerical value obtained in ACT33 is equal to or more than a specified value", or "the numerical value obtained in ACT33 is greater than a specified value". The above condition may be determined as appropriate by the creator of the cart terminal program PRA, etc. Then, if the processor 101 determines YES as there is a possibility of purchase, the process proceeds to ACT34.

[0045] Thus, when the processor 101 judges YES in ACT33, it detects that the recognized product is unlikely to be selected by the customer as the product to be traded. In other words, the processor 101 executes information processing based on the cart terminal program PRA, and the computer having the processor 101 as its central part functions as a detection means.

[0046] As ACT34, the processor 101 performs a confirmation process. This confirmation process is a process for confirming whether or not to register the recognized commodity as a trading commodity. The processor 101, for example, causes a confirmation screen to be displayed on the touch panel 104. The confirmation screen is a GUI (graphical user interface) screen for allowing the customer to specify whether or not to register the recognized commodity as a trading commodity. The contents of the confirmation process may be appropriately determined by, for example, the creator of the cart terminal program PRA, such as outputting a voice message or a warning sound from the sound unit 105 in addition to displaying the confirmation screen.

[0047] FIG. 6 is a diagram showing an example of the confirmation screen. The confirmation screen shown in Figure 6 is a screen in which a confirmation window WIA is superimposed on a registration screen. The confirmation window WIA represents the character strings CSA and CSB and the buttons BUB and BUC. The character string CSA represents a text message to prompt the customer to confirm the recognition result. The character string CSB represents the product name of the recognized product. The button BUB is a soft key that allows the customer to specify registration as a transaction product. The button BUC is a soft key that allows the customer to specify not to register as a transaction product. The contents of the confirmation screen may be determined as appropriate, for example, by the creator of the cart terminal program PRA.

[0048] The customer checks the product name displayed by the character string CSB on the confirmation screen to confirm whether it is the same as the product previously placed in the basket section 930. If it is the same, the customer specifies registration by a predetermined operation such as tapping the button BUB. If it is not the same, the customer specifies not to register by a predetermined operation such as tapping the button BUC.

[0049] In ACT 35, the processor 101 checks whether registration has been designated. If registration has been designated by the customer as described above, the processor 101 determines that the result is YES, and proceeds to ACT 36. When the processor 101 judges YES in ACT35, it is determined that the recognized product is the product selected by the customer. In other words, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as a reception means.

[0050] For example, when the visitor code is stored in the main storage unit 102 or the auxiliary storage unit 103, the processor 101 determines NO in ACT31 as the membership code has not been acquired, and proceeds to ACT36, skipping ACT32 to ACT35. In other words, if the processor 101 has not been able to identify the customer, it proceeds to ACT36 regardless of the possibility that the customer will purchase the recognized product.

[0051] Furthermore, if the processor 101 determines NO in ACT 33 because it is not possible to confirm that the possibility of purchase is low, it skips ACT 34 and ACT 35 and proceeds to ACT 36. In other words, if the processor 101 determines that the possibility of purchase is not low, it proceeds to ACT 36 without confirming whether to register.

[0052] As ACT36, the processor 101 registers the recognized product as a transacted product. That is, the processor 101 updates the transaction data DAA, for example, to include product information of the recognized product as product information of the transacted product. The processor 101 includes at least a product code as a product identifier in the product information. The processor 101 may also include a product name and unit price in the product information. Here, the type of information that the processor 101 includes in the product information may be appropriately determined, for example, by the creator of the cart terminal program PRA.

[0053] When the processor 101 judges YES in ACT35 and executes ACT36, it confirms and registers the recognized commodity as the transaction commodity in accordance with the customer's designation. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer having the processor 101 as its central part functions as confirmation means and registration means.

[0054] In ACT 37, the processor 101 updates the registration screen. For example, the processor 101 updates the registration screen displayed on the touch panel 104 to show an updated list that includes the trading product registered in ACT 36. In ACT38, the processor 101 performs a recognition end process. The recognition end process is a process for making it possible to confirm that recognition has not started for the next product that enters the shooting range of the camera 199. For example, the processor 101 sets the recognition in progress flag to a state indicating that recognition is not in progress.

[0055] In ACT39, the processor 101 waits for the product that has been recognized as described above to complete passing through the shooting range of the camera 199. For example, while the processor 101 is successful in detecting the product area by processing similar to ACT17 in Fig. 3, the processor 101 repeats processing similar to ACT16 to ACT18 at each acquisition timing. Then, when the processor 101 is no longer able to detect the product area, it determines that the passing has been completed, and returns to the standby state of ACT14 and ACT15 in Fig. 3.

[0056] While one transaction commodity passes through the shooting range of camera 199, the acquisition timing may arrive multiple times. The state of the commodity reflected in the image represented by the image data acquired at these multiple acquisition timings changes. For this reason, in the recognition process at ACT22 while processor 101 repeats ACT16 and subsequent steps each time an acquisition timing arrives, even if the commodity was not recognized at the previous acquisition timings, the commodity may be recognized at the latest acquisition timing. In this case, processor 101 executes ACT31 and subsequent steps in FIG. 4 in the same manner as described above.

[0057] However, in cases where a customer holds a commodity with most of the commodity covered by his / her hand, processor 101 may not be able to recognize the commodity in the recognition process in ACT22 in Fig. 3 at any acquisition timing while one transaction commodity passes through the shooting range of camera 199. If processor 101 proceeds to ACT19 in response to the commodity area being unable to be detected in ACT17 because the commodity has left the shooting range of camera 199, processor 101 may determine YES as the commodity is being recognized, for example, based on the recognition in progress flag indicating that the commodity is being recognized, and proceed to ACT40 in Fig. 4. Furthermore, if the processor 101 confirms in the confirmation process in ACT34 in FIG. 4 that the customer has specified not to register, it judges NO in ACT35 and proceeds to ACT40.

[0058] The processor 101 executes the recognition error process as ACT40. The recognition error process is a process for informing a customer that the recognition of the product that was just placed in the basket section 930 has failed. The processor 101, for example, displays a recognition error screen on the touch panel 104. The recognition error screen is a screen for making the customer realize that the product that was just placed in the basket section 930 could not be recognized. The recognition error screen is assumed to be, for example, a screen that displays a text message for making the customer realize that the product that was just placed in the basket section 930 could not be recognized. The contents of the recognition error screen may be appropriately determined, for example, by the creator of the cart terminal program PRA. In addition, at this time, the processor 101 may appropriately combine and execute a notification operation other than the screen display together with the display of the recognition error screen. The notification operation other than the screen display is, for example, an alarm sound or a voice message output from the sound unit 105. The notification operation other than the screen display may be, for example, the lighting or blinking of an alarm lamp not shown in FIG. 1. The notification operation other than the screen display may be, for example, generation of vibration by a vibrator not shown in Fig. 1. The contents of the recognition error processing may be appropriately determined by, for example, the creator of the cart terminal program PRA.

[0059] In ACT41, the processor 101 performs recognition end processing similar to ACT38. In ACT42, the processor 101 displays a registration screen on the touch panel 104. That is, the processor 101 displays a registration screen showing a list of trading products that have been registered at this point in time on the touch panel 104. Then, the processor 101 returns to the standby state of ACT14 and ACT15 in FIG.

[0060] When the customer learns from the recognition error screen that recognition has failed, he or she temporarily removes the product from the basket section 930, and after the registration screen is displayed in ACT42 in Fig. 4, places the product back into the basket section 930. At this time, the processor 101 attempts to recognize the product again in ACT14, ACT16 to ACT18, and ACT20 and thereafter in Fig. 3. At this time, the state in which the product is reflected in the image obtained by the camera 199 is different from the previous time, so there is a possibility that the product will be recognized correctly.

[0061] Once the purchaser has finished registering the purchased items, the purchaser performs a predetermined operation to specify the payment, for example, on touch panel 104. In response, processor 101 determines YES in ACT15 in FIG. 3 and proceeds to payment processing. The payment processing may be the same as that performed in existing cart POS systems, for example, and is not shown in the figures. For example, processor 101 performs processing to transfer transaction data to payment device 200 as payment processing. Alternatively, if a payment terminal (not shown) is attached to cart 900 and the payment terminal is connected to cart terminal 100, the payment processing may be processing to perform credit card payment, code payment, or the like using this payment terminal.

[0062] Thus, in the transaction processing system 1, when the product recognized by the recognition process is a product that the customer is unlikely to purchase, the cart terminal 100 inquires the customer as to whether to register the product as a transaction product, and registers it as a transaction product if the customer specifies registration. This makes it possible to prevent a mistakenly recognized product from being registered as a transaction product.

[0063] In addition, the cart terminal 100 detects that the recognized product is unlikely to be selected based on the customer's purchasing history for all products in the category to which the recognized product belongs, and can therefore detect the low probability of selection while taking into account the purchasing tendencies of each customer.

[0064] This embodiment can be modified in various ways as follows. The processor 101 may also identify the user using other personal identification techniques, such as facial recognition technology.

[0065] It may also be used in combination with another merchandise registration device or transaction processing device, and realized as a merchandise recognition device whose main function is to recognize merchandise.

[0066] The results of the recognition of an item may be used for any purpose other than the registration of a transaction item for transaction processing.

[0067] The present invention may also be realized as a device for recognizing items other than merchandise.

[0068] In the above embodiment, a part of the processing executed by processor 101 may be executed by POS server 500 or any other one or more information processing devices. For example, cart terminal 100 may perform only user interface operations, and various information processing for identifying the transaction commodity may be executed by a computer provided in POS server 500. In other words, various information processing such as identification means, recognition means, detection means, acceptance means, confirmation means, and registration means may be provided in any information processing device such as POS server 500 other than cart terminal 100 included in transaction processing system 1, or may be appropriately distributed among multiple information processing devices included in transaction processing system 1.

[0069] Each function realized by the processor 101 through information processing can be realized in part or in whole by hardware that executes information processing not based on a program, such as a logic circuit, etc. Also, each of the above functions can be realized by combining the above hardware, such as the logic circuit, with software control.

[0070] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0071] 1...transaction processing system, 2...communication network, 100...cart terminal, 101...processor, 102...main memory unit, 103...auxiliary memory unit, 104...touch panel, 105...sound unit, 106...interface unit, 107...wireless communication unit, 108...transmission path, 199...camera, 200...accounting machine, 300...attendant terminal, 400...management terminal, 500...POS server, 900...cart, 910...caster section, 920...handle frame section, 930...basket section, 940...support section, 950...battery case.

Claims

1. An identification means for identifying a user; A recognition means for recognizing an object based on features of the object appearing in an image; a detection means for detecting that the user identified by the identification means is unlikely to select the article recognized by the recognition means; a receiving means for receiving a designation that the item recognized by the recognizing means is the item selected by the user identified by the identifying means in response to the detection means detecting that the item is unlikely to be selected by the user; a determination means for determining, when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means, the item recognized by the recognition means is an item selected by the user identified by the identification means; An article recognition system comprising:

2. The detection means obtains a selection frequency by the user identified by the identification means for all articles belonging to a class to which the article recognized by the recognition means belongs, based on a selection history of the article for the user identified by the identification means, and detects that the possibility of selection is low based on the selection frequency. The item recognition system according to claim 1 .

3. The recognition means recognizes the item selected as a trading commodity by the user identified by the identification means based on an image of the item, The detection means detects that the user identified by the identification means is unlikely to select the product as a trading product. The article recognition system according to claim 1 or 2.

4. A computer provided in the item recognition system, An identification means for identifying a user; A recognition means for recognizing an object based on features of the object appearing in an image; a detection means for detecting that the user identified by the identification means is unlikely to select the article recognized by the recognition means; a receiving means for receiving a designation that the item recognized by the recognizing means is the item selected by the user identified by the identifying means in response to the detection means detecting that the item is unlikely to be selected by the user; a determination means for determining, when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means, the item recognized by the recognition means is an item selected by the user identified by the identification means; An information processing program that enables the system to function as such.

5. An identification means for identifying a user; a recognition means for recognizing an item selected as a trading commodity by a user identified by the identification means based on features of the item appearing in an image of the item; a detection means for detecting that the user identified by the identification means is unlikely to select the article recognized by the recognition means; a receiving means for receiving a designation that the item recognized by the recognizing means is the item selected by the user identified by the identifying means in response to the detection means detecting that the item is unlikely to be selected by the user; a determination means for determining, when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means, the item recognized by the recognition means is an item selected by the user identified by the identification means; a registration means for registering the item confirmed by the confirmation means as a trading item; A transaction processing system comprising:

Images