Cart terminal and information processing program

Abstract

To prevent a wrong recognition result from being used.SOLUTION: The item recognition system according to an embodiment includes identification means, recognition means, detection means, reception means, and confirmation means. The identification means identifies a user. The recognition means recognizes an item represented in an image on the basis of characteristics of the item. The detection means detects a low possibility that the identified person selects the recognized item. The reception means receives the specification that the recognized item is an item that the identified user selected, in response to detection that the possibility of the selection is low made by the selection means. The confirmation means confirms that the recognized item is an item that the identified user selected, in a case where that a low possibility of the selection is not detected and in a case where the specification that the recognized item is an item that the identified user selected is received.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] Embodiments of the present invention are: Cart terminal and Information processing program Mu Regarding. [Background technology]

[0002] A registration system has already been proposed for display-type retail stores that recognizes items placed in a customer's shopping cart from an image and automatically registers them as items the customer will purchase. POS systems using such registration systems are sometimes referred to as frictionless POS systems. However, since multiple separate products can sometimes look similar, it was difficult to completely avoid misidentification based solely on the features shown in the images. Given these circumstances, measures to address such misconceptions were needed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-150436 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to prevent erroneous recognition results from being used. Cart terminal and Information processing program M The purpose is to provide. [Means for solving the problem]

[0005] The article recognition system of this embodiment comprises identification means, recognition means, detection means, reception means and confirmation means. The identification means identifies the user. The recognition means, I photographed the items. Appearing in the image The relevantThe object is recognized based on the characteristics of the object. Based on the item selection history of the user identified by the identification means, the selection frequency by the user identified by the identification means is determined for all items belonging to the classification to which the item recognized by the recognition means belongs, and based on that selection frequency, The recognition means detects that the user identified by the identification means is unlikely to select the item recognized by the recognition means. The acceptance means, in response to the detection means detecting the low possibility of selection, receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means. The confirmation means confirms the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect the low possibility of selection and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a block diagram showing a schematic configuration of a transaction processing system according to an embodiment and a main circuit configuration of a cart terminal. [Figure 2] FIG. 10 is a perspective view showing an example of a state in which a cart terminal is attached to a cart. [Figure 3] Flowchart of the registration process. [Figure 4] 10 is a flowchart of a registration process. [Figure 5] FIG. 10 is a diagram illustrating an example of a registration screen. [Figure 6] FIG. 10 is a diagram illustrating an example of a confirmation screen. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment will be described below with reference to the drawings. This embodiment will describe an example of a transaction processing system that incorporates the functions of an item recognition system. FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system 1 according to this embodiment and a main circuit configuration of a cart terminal 100. As shown in FIG.

[0008] Transaction processing system 1 is an information processing system for processing transactions involving the sale of goods to customers in a store. Transaction processing system 1 is configured so that a cart terminal 100, a checkout machine 200, an attendant terminal 300, a management terminal 400, and a POS server 500 can communicate with each other via a communication network 2.

[0009] Communication network 2 can use the internet, VPN (virtual private network), LAN (local area network), public communication network, mobile communication network, etc., individually or in appropriate combinations. As an example, communication network 2 can be used by combining a LAN and the internet. Note that the transaction processing system 1 may contain any number of cart terminals 100, accounting machines 200, attendant terminals 300, management terminals 400, and POS servers 500, but Figure 1 shows only one of each.

[0010] The cart terminal 100 is an information processing terminal attached to a shopping cart (hereinafter referred to as "cart") provided in the store. The cart terminal 100 is lent to customers along with the cart when they purchase goods in the store. The cart terminal 100 is a terminal device that, upon receiving an operation from the customer, performs a registration process to register the goods that the customer intends to purchase as transaction goods. As described later, the cart terminal 100 performs a process to recognize the goods to be registered as transaction goods from an image in which the goods are captured. In other words, the cart terminal 100 has the function of a product recognition device. Goods are an example of articles.

[0011] The accounting machine 200 is installed in the store and performs accounting processing related to the accounting and settlement of the price of the traded goods registered by the cart terminal 100. The accounting machine 200 is operated by an operator during the accounting process. The operator of the accounting machine 200 is mainly a customer. In some cases, a store employee may operate the accounting machine 200. For example, the accounting machine 200 can be the same one used in an existing cart POS system.

[0012] The attendant terminal 300 is an information processing terminal operated by a store clerk. The attendant terminal 300 is a terminal device for a user interface related to information processing to support the work of the store clerk regarding transactions processed by the transaction processing system 1. The work of the store clerk, for example, involves monitoring the implementation status of transactions in progress and providing appropriate support to customers. As the basic hardware of the attendant terminal 300, various general-purpose information processing devices such as desktop, notebook, tablet, and mobile types can be used. The attendant terminal 300 can be realized by installing an information processing program for realizing the various functions of the attendant terminal 300 on a computer equipped with such basic hardware.

[0013] Management terminal 400 is an information terminal that serves as a user interface for an administrator who manages transaction processing system 1. Various general-purpose information processing devices, such as desktop, notebook, tablet, and mobile types, can be used as the basic hardware of management terminal 400. Management terminal 400 can be realized by installing an information processing program for realizing the various functions of management terminal 400 on a computer equipped with such basic hardware.

[0014] The POS server 500 is an information processing device that executes management processes for managing product purchase and sale transactions that are processed through registration processes in the cart terminal 100 and transaction processes in the payment device 200. An appropriate information processing device such as a general-purpose server device can be used as the basic hardware of the POS server 500. The POS server 500 can be realized by installing an information processing program for realizing the various functions of the POS server 500 on a computer equipped with such basic hardware.

[0015] The cart terminal 100 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, a touch panel 104, a sound unit 105, an interface unit 106, a wireless communication unit 107, and a transmission path 108. The processor 101, the main memory unit 102, the auxiliary memory unit 103, the touch panel 104, the sound unit 105, the interface unit 106, and the wireless communication unit 107 are capable of communicating with each other via the transmission path 108.

[0016] By connecting the processor 101, the main memory unit 102, and the auxiliary memory unit 103 with a transmission line 108, a computer is configured to perform information processing for controlling the cart terminal 100. The processor 101 corresponds to the central part of the computer described above. The processor 101 performs information processing to control each part in order to realize various functions as a cart terminal 100, in accordance with information processing programs such as the operating system and application programs.

[0017] The main memory unit 102 corresponds to the main memory portion of the computer. The main memory unit 102 includes a read-only memory area and a rewritable memory area. The main memory unit 102 stores part of the information processing program in the read-only memory area. The main memory unit 102 may also store data required for the processor 101 to execute processes for controlling each component in the read-only memory area or the rewritable memory area. The main memory unit 102 uses the rewritable memory area as a work area for the processor 101.

[0018] The auxiliary storage unit 103 corresponds to the auxiliary storage portion of the computer. The auxiliary storage unit 103 may be, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), an SSD (solid state drive), or any other well-known storage device. The auxiliary storage unit 103 stores data used by the processor 101 when performing various processes and data generated by the processes performed by the processor 101. The auxiliary storage unit 103 may also store the information processing program. In this embodiment, the auxiliary storage unit 103 stores a cart terminal program PRA, which is one of the information processing programs. The cart terminal program PRA is an application program that describes the registration process procedures. A portion of the storage area of ​​the auxiliary storage unit 103 is used as an area for storing transaction data DAA. The transaction data DAA is data representing the contents of one transaction.

[0019] The touch panel 104 displays a screen for presenting information to the operator, and also inputs instructions by the operator touching the screen through the touch panel 104. The sound unit 105 outputs sounds for various guidance and warnings. As the sound unit 105, various well-known sound devices such as a voice synthesis device and a buzzer can be used alone or in combination.

[0020] An external device such as a camera 199 is connected to the interface unit 106. The interface unit 106 acts as an interface for sending and receiving data to and from the connected external device. An existing USB (universal serial bus) controller or the like can be used as the interface unit 106. The camera 199 may repeat photographing at regular intervals, or may photograph in response to instructions from the processor 101. A camera built into the cart terminal 100 may be used instead of the camera 199.

[0021] The wireless communication unit 107 performs communication processing for wireless data communication via the communication network 2. For example, an existing wireless communication device for a wireless LAN can be used as the wireless communication unit 107. Alternatively, a communication unit connected to the communication network 2 via a wired connection may be used instead of, or in addition to, the wireless communication unit 107. The transmission path 108 includes an address bus, a data bus, and control signal lines, and transmits data and control signals exchanged between the connected components.

[0022] The basic hardware of the cart terminal 100 is expected to be, for example, the hardware of a tablet-type information processing device. Generally, the cart terminal 100 is transferred with the cart terminal program PRA stored in the auxiliary storage unit 103. However, the hardware and the cart terminal program PRA may be transferred separately, either without the cart terminal program PRA stored in the auxiliary storage unit 103, or with a different version of the same type of application program stored in the auxiliary storage unit 103. Furthermore, the cart terminal 100 may be configured by writing the cart terminal program PRA to the auxiliary storage unit 103 in response to an operation by any worker. The cart terminal program PRA can be transferred by recording it on a removable recording medium such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory, or by communication over a network.

[0023] FIG. 2 is a perspective view showing an example of how the cart terminal 100 is attached to the cart. The cart 900 includes a caster portion 910 , a handle frame portion 920 , a basket portion 930 , a support portion 940 , and a battery case 950 . The caster section 910 has four wheels 911 for smoothly moving the cart 900 on the floor. The wheels 911 are mounted on the frame 912 so as to be rotatable around a vertical axis.

[0024] The handle frame portion 920 includes a pair of vertical frames 921, 921 and a handle bar 922. The vertical frames 921, 921 are erected above the two wheels of the caster portion 910. The handle bar 922 connects the upper ends of the vertical frames 921, 921 together. The basket section 930 is provided horizontally from a midpoint of the handle frame section 920. The basket section 930 is shaped like a basket with an opening at the top that allows products to be placed inside, and functions as a container for storing products that customers wish to purchase. A camera 199 is attached to the upper end of the basket section 930, closer to the handlebar 922, in a state that allows it to photograph the opening of the basket section 930 from the side. The photographing range of the camera 199 is determined so that the products placed inside the basket section 930 are captured through the opening of the basket section 930.

[0025] The support unit 940 includes a pole 941. The pole 941 is attached to one of the vertical frames 921 so that its tip is positioned higher than the handlebars 922. The cart terminal 100 described above is attached to the tip of the pole 941. As a result, the support unit 940 supports the cart terminal 100 in the state shown in FIG. The battery case 950 is attached between the vertical frames 921, 921 on the lower end side of the handle frame portion 920. The battery case 950 houses a battery used as an external power source for the cart terminal 100.

[0026] Next, the operation of the transaction processing system 1 configured as described above will be described. Note that the contents of the various processes described below are merely examples, and it is possible to change the order of some of the processes, omit some of the processes, or add other processes as appropriate. For example, in the following description, in order to clearly explain the characteristic operations of this embodiment, the description of some of the processes will be omitted. For example, if an error occurs, processing may be performed to deal with the error. Also, for example, in various standby states, the system may wait for the occurrence of an event different from what is described, and transition to processing according to the event that has occurred.

[0027] Customers of a retail store, that is, customers who intend to purchase goods displayed and sold at the store, perform predetermined operations to start using the unused shopping cart terminal 100. Once these operations are performed, the processor 101 starts the registration process based on the shopping cart terminal program PRA.

[0028] 3 and 4 are flowcharts of the registration process. In Figure 3, the processor 101, as ACT11, performs the login process. For example, the processor 101 displays a login screen on the touch panel 104 for the customer to specify a user code and password, and receives the user code and password from the customer. Then, for example, the processor 101 authenticates whether the customer is a registered user based on the user code and password specified by the customer, using a well-known user authentication technology. If authentication is successful, the processor 101 stores the specified user code as the customer's identification code, for example, in the main storage unit 102 or the auxiliary storage unit 103. Alternatively, the cart terminal 100 may be equipped with a reader that reads the user code recorded on the user card, and the processor 101 may perform user authentication using the user code when the user code is read by this reader, or it may store the user code as the customer's identification code without performing user authentication. The processor 101 also allows logins that do not involve obtaining a user code. In other words, the processor 101, for example, receives a visitor's designation on the login screen, and instead of obtaining a user code in response to that designation, stores a predetermined visitor code as the customer's identification code in, for example, the main storage unit 102 or the auxiliary storage unit 103.

[0029] As ACT12, the processor 101 generates new transaction data DAA for the target transaction. Specifically, the processor 101 determines a new transaction code, for example, that is different from the transaction code used as an identifier to identify other transactions, according to predetermined rules, generates new transaction data DAA that includes this transaction code but does not include product information about the traded product, and stores it in the auxiliary storage unit 103. In ACT13, the processor 101 displays a registration screen on the touch panel 104. The registration screen is a screen for allowing the customer to recognize the registration status of the transaction product.

[0030] FIG. 5 is a diagram showing an example of the registration screen. Note that in the screen examples shown in FIG. 5 and other figures, some display objects may be omitted from the illustration. The registration screen shown in Figure 5 has an image representing the registration status placed in the display area ARA. The registration screen is represented by the button BUA. In the example in Figure 5, the image placed in the display area ARA shows that two items have been registered, one each with product names "AAAAAAAAAA" and "BBBBBBBBBB", with unit prices of 178 yen and 198 yen respectively, and their reference price is 406 yen. The reference price is calculated by subtracting the discount amount from various services such as coupon services from the sum of the unit prices of all transaction items. If the transaction is settled without any changes to the transaction items or applicable services, this reference price will be the settlement amount. The image placed in the display area ARA changes sequentially according to the registration status of the transaction items. However, when ACT13 is executed, the transaction items have not yet been registered, so the display area ARA at this time does not show information on each item, but displays an image indicating a total of 0 items and a reference price of 0 yen. The button BUA is a soft key for the operator to instruct the payment process. The contents of the registration screen may be determined as appropriate by, for example, the creator of the cart terminal program PRA.

[0031] As ACT14, processor 101 checks if it is time to acquire an image. If processor 101 cannot confirm the event, it determines NO and proceeds to ACT15. As ACT15, processor 101 checks if accounting has been requested. If processor 101 cannot confirm the relevant event, it determines NO and returns to ACT14. Thus, processor 101 waits for either the acquisition timing to occur or accounting to be specified as ACT14 and ACT15.

[0032] The acquisition timing is expected to be, for example, immediately after progressing from ACT13 to ACT14, and at regular time intervals thereafter. The acquisition timing may be determined as appropriate by, for example, the creator of the cart terminal program PRA. However, the acquisition timing should be determined so that it occurs at least once, preferably multiple times, while the product to be placed in the basket section 930 is passing through the shooting range of the camera 199.

[0033] When it's time to acquire the data, processor 101 determines YES in ACT14 and proceeds to ACT16. As ACT16, the processor 101 acquires image data output by the camera 199. For example, if the camera 199 is taking pictures at regular intervals, the processor 101 acquires the image data output by the camera 199 in accordance with the first picture taken after the acquisition timing. For example, if the camera 199 takes pictures in response to instructions from the processor 101, the processor 101 instructs the camera 199 to take a picture, and acquires the image data output by the camera 199 in response. The processor 101 then stores the acquired image data in the main storage unit 102 or the auxiliary storage unit 103.

[0034] As ACT 17, the processor 101 detects an area in which a product is reflected (hereinafter referred to as a product area) from the image represented by the acquired image data. The area to be detected here can be determined using, for example, deep learning using a convolutional neural network (CNN) or a well-known method using graph cuts, and the method is not limited.

[0035] In ACT 18, the processor 101 checks whether the product area was successfully detected. If the product area could not be detected in ACT 17, the processor 101 determines NO and proceeds to ACT 19. Thus, the processor 101 proceeds to ACT 19 when the product is not captured in the image captured by the camera 199.

[0036] In ACT19, the processor 101 checks whether the product is being recognized. If the processor 101 cannot confirm this event as described below, it determines "NO" and returns to the standby state of ACT14 and ACT15. The customer searches for the product to be purchased in the store. The customer then removes the product they wish to purchase from the sales floor and places it in the basket section 930. At this time, the product enters the camera 199's shooting range and is captured in the image represented by the image data output by the camera 199. In other words, until the customer attempts to place the product in the basket section 930, the processor 101 does not detect the product area in ACT17 even when the acquisition timing arrives, and product recognition has not yet begun. Therefore, it determines "NO" in both ACT18 and ACT19, and returns to the standby state of ACT14 and ACT15, repeating this process. When the product the customer is attempting to place in the basket section 930 is captured in the image, the processor 101 detects the product area in ACT17. In this case, the processor 101 determines YES in ACT18 and proceeds to ACT20.

[0037] As ACT20, processor 101 checks whether the product is being recognized. If processor 101 cannot confirm the relevant event as described later, it determines NO and proceeds to ACT21. As ACT21, processor 101 performs recognition start processing. Recognition start processing is a process to confirm that the product that has entered the shooting range of camera 199 is being recognized. For example, processor 101 sets the recognition flag to a state that indicates that recognition is in progress.

[0038] As ACT22, processor 101 performs recognition processing. Recognition processing is the process of recognizing products that are captured in the product area based on the image of the product area detected in ACT17. This recognition processing may be either product recognition based on the appearance of the product, product recognition based on the product barcode displayed on the product, or both. For product recognition based on the appearance of the product, it is expected that deep learning using CNN, a method suitable for image recognition, will be used. Alternatively, for product recognition based on the appearance of the product, the method may be limited to using OCR (optical character recognition) or feature point matching. Thus, by processor 101 executing information processing based on the cart terminal program PRA, the computer with processor 101 as its central component functions as a recognition means. Furthermore, if product recognition based on the product barcode displayed on the product is applied to the recognition processing, another device such as a laser scanner that detects the pattern of the product barcode without relying on an image may be used instead of or in addition to camera 199.

[0039] In ACT 23, the processor 101 checks whether the recognition in ACT 21 was successful. If the recognition process does not succeed in recognizing the product, the processor 101 determines NO in ACT 23 and returns to the standby state of ACT 14 and ACT 15. In other words, even if the product is captured in the image, if the processor 101 is unable to recognize the product from the image, the processor 101 returns to the standby state of ACT 14 and ACT 15.

[0040] If the processor 101 proceeds to ACT20 again because the same product remains in the image at the subsequent acquisition timing, it will judge YES, for example, depending on whether the recognition in progress flag indicates that recognition is in progress, and will skip ACT21 and proceed to ACT22.

[0041] Now, if the processor 101 has successfully recognized the device in ACT22, it will determine YES in ACT23 and proceed to ACT31 in Figure 4. In ACT 31, the processor 101 checks whether the membership code has been acquired. If the membership code acquired in ACT 11 is stored in the main storage unit 102 or the auxiliary storage unit 103, the processor 101 determines that the membership code has been acquired (YES) and proceeds to ACT 32. Note that if the processor 101 acquires the membership code in ACT 11, the processor 101 may set an acquisition flag to a state indicating that the membership code has been acquired, and make a determination in ACT 31 based on the acquisition flag.

[0042] In ACT32, the processor 101 executes a possibility determination process. The possibility determination process is a process for determining the possibility that a customer identified by the acquired membership code will purchase the product recognized in ACT21 in FIG. 3 (hereinafter referred to as the recognized product). As an example, the processor 101 refers to information such as attribute information or purchase history of the customer identified by the membership code to determine a numerical value representing the possibility that the customer will purchase the recognized product. More specifically, for example, the processor 101 determines the product category to which the recognized product belongs, determines the customer's purchase frequency for all products belonging to that product category based on the purchase history, and uses this purchase frequency as a numerical value representing the possibility. That is, as an example, it is assumed that the recognized product belongs to a product category called "cosmetics for women." In this case, the processor 101 determines the purchase frequency for all products belonging to the product category "cosmetics for women" from the purchase history of the customer identified by the acquired membership code, and uses this numerical value as a numerical value representing the possibility. In addition, as an example, when the recognized product belongs to the product category of "cosmetics for women" and the gender included in the customer's attribute information is "male," the processor 101 reduces the numerical value representing the possibility.

[0043] The processor 101 may determine the purchase history from purchase history data managed in a well-known manner by, for example, the POS server 500 or a server separately provided for managing purchase history. The processor 101 may use, for example, attribute information managed in a well-known manner by the POS server 500 or a server separately provided for member management, for customer attribute information.

[0044] In ACT33, the processor 101 checks whether the likelihood of a purchase is low based on the numerical value representing the likelihood determined as described above. For example, the processor 101 determines that there is a possibility of a purchase if the numerical value determined in ACT33 satisfies a predetermined condition. The condition here is assumed to be defined as, for example, "the numerical value determined in ACT33 is equal to or less than a specified value." However, the condition may also be, "the numerical value determined in ACT33 is less than a predetermined value." Alternatively, the numerical value determined in ACT33 is smaller as the likelihood increases, and the condition may be, for example, "the numerical value determined in ACT33 is equal to or greater than a specified value," or "the numerical value determined in ACT33 is greater than a specified value." The above conditions may be appropriately determined by, for example, the creator of the cart terminal program PRA. If the processor 101 determines YES, indicating that there is a possibility of a purchase, the processor 101 proceeds to ACT34.

[0045] Thus, when the processor 101 determines YES in ACT 33, it has detected that the recognized product is unlikely to be selected by the customer as the product to be traded. In other words, by the processor 101 executing information processing based on the cart terminal program PRA, the computer with the processor 101 as its central part functions as a detection means.

[0046] As ACT34, the processor 101 performs a confirmation process. This confirmation process is a process for confirming whether or not to register the recognized product as a trading product. The processor 101, for example, displays a confirmation screen on the touch panel 104. The confirmation screen is a GUI (graphical user interface) screen that allows the customer to specify whether or not to register the recognized product as a trading product. The content of the confirmation process may be determined appropriately by, for example, the creator of the cart terminal program PRA, such as outputting a voice message or a warning sound from the sound unit 105 in addition to displaying the confirmation screen.

[0047] Figure 6 shows an example of a confirmation screen. The confirmation screen shown in Figure 6 is a screen in which a confirmation window WIA is overlaid on the registration screen. The confirmation window WIA represents the strings CSA, CSB, and buttons BUB, BUC. The string CSA represents a text message prompting the customer to confirm the recognition result. The string CSB represents the product name of the recognized product. Button BUB is a soft key for the customer to specify that the product be registered as a transaction product. Button BUC is a soft key for the customer to specify that the product not be registered as a transaction product. The content of the confirmation screen may be determined as appropriate by, for example, the creator of the shopping cart terminal program PRA.

[0048] The customer checks the product name, represented by the string CSB on the confirmation screen, to see if it is the same as the product they just placed in the shopping cart 930. If it is the same, the customer specifies registration by performing a predetermined operation, such as tapping button BUB. If it is not the same, the customer specifies not to register by performing a predetermined operation, such as tapping button BUC.

[0049] In ACT 35, the processor 101 checks whether registration has been designated. If registration has been designated by the customer as described above, the processor 101 determines the answer as YES and proceeds to ACT 36. When processor 101 determines YES in ACT35, it receives confirmation that the recognized product is the product selected by the customer. In other words, by having processor 101 perform information processing based on the cart terminal program PRA, the computer with processor 101 as its central component functions as a reception mechanism.

[0050] Furthermore, if, for example, the visitor code is stored in the main memory unit 102 or the auxiliary memory unit 103, the processor 101 determines NO in ACT31, indicating that the member code has not been obtained, and proceeds to ACT36, skipping ACT32 to ACT35. In other words, if the processor 101 cannot identify the customer, it proceeds to ACT36 regardless of the likelihood that the customer will purchase the recognized product.

[0051] Furthermore, if processor 101 determines NO in ACT33 because it cannot confirm that the likelihood of purchase is low, it skips ACT34 and ACT35 and proceeds to ACT36. In other words, if processor 101 does not determine that the likelihood of purchase is low, it proceeds to ACT36 without confirming whether to register.

[0052] As ACT36, the processor 101 registers the recognized product as a transaction product. That is, the processor 101 updates the transaction data DAA to include product information of the recognized product as product information of the transaction product. The processor 101 includes at least a product code as a product identifier in the product information. The processor 101 may also include the product name and unit price in the product information. Here, the type of information that the processor 101 includes in the product information may be determined as appropriate by, for example, the creator of the cart terminal program PRA.

[0053] When the processor 101 judges YES in ACT 35 and executes ACT 36, it confirms and registers the recognized product as the transaction product in accordance with the customer's designation. Thus, by the processor 101 executing information processing based on the cart terminal program PRA, the computer with the processor 101 as its central part functions as confirmation means and registration means.

[0054] As ACT37, processor 101 updates the registration screen. For example, processor 101 updates the registration screen to display on touch panel 104 a list that includes the trading products registered in ACT36. As ACT38, processor 101 performs recognition termination processing. Recognition termination processing is a process to confirm that recognition of the next product to enter the shooting range of camera 199 has not yet begun. For example, processor 101 sets the recognition in progress flag to a state that recognition is not in progress.

[0055] In ACT 39, the processor 101 waits for the product that has been recognized as described above to complete passing through the shooting range of the camera 199. For example, while the processor 101 is successful in detecting the product area by processing similar to ACT 17 in Fig. 3, the processor 101 repeats processing similar to ACT 16 to ACT 18 at each acquisition timing. Then, when the processor 101 can no longer detect the product area, it determines that the passing has been completed, and returns to the standby state of ACT 14 and ACT 15 in Fig. 3.

[0056] While a single transaction commodity passes through the imaging range of camera 199, acquisition timings may occur multiple times. The state of the commodity reflected in the image represented by the image data acquired at these multiple acquisition timings changes. Therefore, in the recognition process at ACT 22, while processor 101 repeats ACT 16 and subsequent steps each time an acquisition timing occurs, even if the commodity was not recognized at the previous acquisition timings, the commodity may be recognized at the latest acquisition timing. In this case, processor 101 executes ACT 31 and subsequent steps in FIG. 4 in the same manner as described above.

[0057] However, in cases such as when a customer holds a product with most of the product covered by their hand, processor 101 may not be able to recognize the product in the recognition process in ACT 22 in Fig. 3 at any of the acquisition timings while a single transaction product passes through the shooting range of camera 199. If processor 101 proceeds to ACT 19 because the product area can no longer be detected in ACT 17 due to the product leaving the shooting range of camera 199, processor 101 determines YES as the product is being recognized based on, for example, the recognition in progress flag indicating that recognition is in progress, and proceeds to ACT 40 in Fig. 4. Furthermore, if processor 101 confirms that the customer has specified not to register in the confirmation process in ACT34 in Figure 4, it determines NO in ACT35 and proceeds to ACT40.

[0058] As ACT40, the processor 101 executes recognition error processing. Recognition error processing is a process to inform the customer that the recognition of the item placed in the basket unit 930 immediately before has failed. For example, the processor 101 displays a recognition error screen on the touch panel 104. The recognition error screen is a screen that makes the customer aware that the item placed in the basket unit 930 immediately before could not be recognized. The recognition error screen is expected to be a screen that displays a text message to make the customer aware that the item placed in the basket unit 930 immediately before could not be recognized. The content of the recognition error screen may be determined as appropriate by, for example, the creator of the cart terminal program PRA. At this time, the processor 101 may also execute a notification operation other than screen display in combination with the display of the recognition error screen as appropriate. A notification operation other than screen display may be, for example, an alarm sound or voice message output from the sound unit 105. A notification operation other than screen display may be, for example, the lighting or flashing of an alarm lamp, which is not shown in Figure 1. Other notification actions besides screen displays may include, for example, the generation of vibrations by a vibrator, which is not shown in Figure 1. The content of recognition error handling may be determined as appropriate by, for example, the creator of the cart terminal program PRA.

[0059] In ACT41, the processor 101 performs recognition termination processing in the same manner as in ACT38. As ACT42, processor 101 displays the registration screen on touch panel 104. In other words, processor 101 displays the registration screen on touch panel 104 again, showing a list of traded products that have been registered at this point. After this, processor 101 returns to the standby state of ACT14 and ACT15 in Figure 3.

[0060] When the customer learns from the recognition error screen that recognition has failed, he or she temporarily removes the product from the basket section 930, and after the registration screen is displayed in ACT42 in Fig. 4, places the product back into the basket section 930. At this time, the processor 101 attempts to recognize the product again in ACT14, ACT16 to ACT18, and ACT20 and thereafter in Fig. 3. At this time, the state in which the product is reflected in the image obtained by the camera 199 is different from the previous time, so there is a possibility that the product will be recognized correctly.

[0061] After registering the purchased items, the purchaser performs a predetermined operation to specify the payment method, for example, on the touch panel 104. In response, the processor 101 determines YES in ACT15 in FIG. 3 and proceeds to the payment process. The payment process may be similar to that performed in existing cart POS systems, and is not shown in the figures. For example, the processor 101 performs the payment process by transferring transaction data to the payment device 200. Alternatively, if a payment terminal (not shown) is attached to the cart 900 and the payment terminal is connected to the cart terminal 100, the payment process may be the process for performing credit card payment, code payment, or the like using the payment terminal.

[0062] Thus, in the transaction processing system 1, if the product recognized by the recognition process is a product that the customer is unlikely to purchase, the cart terminal 100 asks the customer whether to register the product as a transaction product, and registers it as a transaction product if the customer specifies registration. This makes it possible to prevent a mistakenly recognized product from being registered as a transaction product.

[0063] Furthermore, the shopping cart terminal 100 detects that the likelihood of selecting a recognized product is low based on the customer's purchase history for all products belonging to the same category as the product in question, thus enabling detection while considering each customer's purchasing tendencies.

[0064] This embodiment can be modified in various ways as follows. The processor 101 may also identify the user using other personal identification techniques, such as facial recognition technology.

[0065] It may also be used in combination with another product registration device or transaction processing device, and realized as a product recognition device whose main function is to recognize products.

[0066] The results of the recognition of an item may be used for any purpose, not just for registering the transaction item for transaction processing.

[0067] It may also be realized as a device for recognizing items other than merchandise.

[0068] In the above embodiment, part of the processing executed by processor 101 may be executed by POS server 500 or any one or more other information processing devices. For example, cart terminal 100 may perform only user interface operations, and various information processing for identifying the transaction item may be executed by a computer provided in POS server 500. In other words, various information processing functions such as identification means, recognition means, detection means, acceptance means, confirmation means, and registration means may be provided in any information processing device other than cart terminal 100 included in transaction processing system 1, such as POS server 500, or may be appropriately distributed among multiple information processing devices included in transaction processing system 1.

[0069] Some or all of the functions realized by the processor 101 through information processing can be realized by hardware that executes information processing not based on a program, such as a logic circuit, etc. Each of the above functions can also be realized by combining hardware such as the above logic circuit with software control.

[0070] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. The invention described in the original claims of this application is listed below. [Note 1] An identification means for identifying the user, Recognition means for recognizing an object based on the characteristics of the object that appears in the image, a detection means for detecting that the user identified by the identification means is unlikely to select the item recognized by the recognition means; In response to the detection means detecting that the likelihood of selection is low, a receiving means receives designation that the item recognized by the recognition means is an item selected by the user identified by the identification means, a determination means for determining the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means; An item recognition system equipped with the following features. [Supplementary Note 2] The detection means calculates the frequency of selection by the user identified by the identification means for all articles belonging to a class to which the article recognized by the recognition means belongs, based on the selection history of the articles for the user identified by the identification means, and detects that the possibility of selection is low based on the selection frequency. The item recognition system described in Appendix 1. [Appendix 3] The recognition means recognizes the item selected as a trading commodity by the user identified by the identification means based on an image of the item, The detection means detects that the user identified by the identification means is unlikely to select it as a trading item. The item recognition system described in Appendix 1 or Appendix 2. [Note 4] The computer included in the object recognition system A means for identifying the user, recognition means for recognizing an object based on features of the object appearing in an image; A detection means for detecting that the item recognized by the recognition means is unlikely to be selected by the user identified by the identification means, In response to the detection means detecting that the likelihood of selection is low, a receiving means receives designation that the item recognized by the recognition means is an item selected by the user identified by the identification means, a determination means for determining the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means; An information processing program that makes it function as such. [Appendix 5] Identification means for identifying a user; Recognition means that recognizes an item based on the characteristics of the item that appear in an image of the item selected as a trading item by a user identified by an identification means, A detection means for detecting that the item recognized by the recognition means is unlikely to be selected by the user identified by the identification means, In response to the detection means detecting that the likelihood of selection is low, a receiving means receives designation that the item recognized by the recognition means is an item selected by the user identified by the identification means, a determination means for determining the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means; a registration means for registering the item confirmed by the confirmation means as a trading commodity; A transaction processing system comprising: [Explanation of symbols]

[0071] 1...Transaction processing system, 2...Communication network, 100...Cart terminal, 101...Processor, 102...Main memory unit, 103...Auxiliary memory unit, 104...Touch panel, 105...Sound unit, 106...Interface unit, 107...Wireless communication unit, 108...Transmission line, 199...Camera, 200...Accounting machine, 300...Attendant terminal, 400...Management terminal, 500...POS server, 900...Cart, 910...Caster section, 920...Handle frame section, 930...Basket section, 940...Support section, 950...Battery case.

Claims

1. A means for identifying the user, A recognition means that recognizes an item based on the characteristics of the item that appear in an image of the item, a detection means for calculating a selection frequency by the user identified by the identification means for all articles belonging to a category to which the article recognized by the recognition means belongs, based on a selection history of the articles related to the user identified by the identification means, and detecting, based on the selection frequency, that the user identified by the identification means is unlikely to select the article recognized by the recognition means; In response to the detection means detecting that the likelihood of selection is low, a receiving means receives designation that the item recognized by the recognition means is an item selected by the user identified by the identification means, a determination means for determining the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means; A cart terminal equipped with [a specific feature / equipment].

2. The recognition means recognizes an item based on an image of the item selected as a transaction item by the user identified by the identification means. The detection means detects that the user identified by the identification means is unlikely to select it as a trading item. The cart terminal according to claim 1.

3. The computer provided in the cart terminal A means for identifying the user, A recognition means that recognizes an item based on the characteristics of the item that appear in an image of the item, a detection means for calculating a selection frequency by the user identified by the identification means for all articles belonging to a category to which the article recognized by the recognition means belongs, based on a selection history of the articles related to the user identified by the identification means, and detecting, based on the selection frequency, that the user identified by the identification means is unlikely to select the article recognized by the recognition means; In response to the detection means detecting that the likelihood of selection is low, a receiving means receives designation that the item recognized by the recognition means is an item selected by the user identified by the identification means, a determination means for determining the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means; An information processing program that makes it function as such.

4. A means for identifying the user, Recognition means that recognizes an item based on the characteristics of the item that appear in an image of the item selected as a trading item by a user identified by an identification means, a detection means for calculating a selection frequency by the user identified by the identification means for all articles belonging to a category to which the article recognized by the recognition means belongs, based on a selection history of the articles related to the user identified by the identification means, and detecting, based on the selection frequency, that the user identified by the identification means is unlikely to select the article recognized by the recognition means; In response to the detection means detecting that the likelihood of selection is low, a receiving means receives designation that the item recognized by the recognition means is an item selected by the user identified by the identification means, a determination means for determining the item recognized by the recognition means as the item selected by the user identified by the identification means when the detection means does not detect that the item is unlikely to be selected and when the acceptance means receives a designation that the item recognized by the recognition means is the item selected by the user identified by the identification means; a registration means for registering the item confirmed by the confirmation means as a trading commodity; A cart terminal equipped with [a specific feature / equipment].

Images