Item identification system, registration system and information processing program

Abstract

To provide an article identification system capable of accurately identifying an article reflected in an image on the basis of the image.SOLUTION: An article identification system of the embodiment includes detection means, article detection means, selection means, and identification means. Key point detection means detects key points of a hand reflected in an appropriate image of a plurality of images photographed by at least one imaging device during a predetermined period. The article detection means detects an article reflected in each of the plurality of images for which the key point detection means detects the key points. The selection means selects, from among the images where an article is detected by the article detection means, an image where the number of key points detected by the key point detection means is smaller. The identification means identifies, on the basis of an image of a region where the article detected by the article detection means is reflected, among the images selected by the selection means, the article.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to an item identification system, a registration system, and an information processing program. [Background technology]

[0002] A registration system has already been proposed for display sales stores that identifies products placed in a shopping cart by a customer from an image taken of the product being held in the hand as it is being placed in the shopping cart, and automatically registers the product as a product to be purchased. POS systems that use such registration systems are sometimes called frictionless POS systems. However, depending on how the customer holds the product, there is a risk that much of the product will be hidden by the hand and not appear in the image, making it impossible to correctly identify. In view of these circumstances, it has been desired to be able to accurately identify an item shown in an image based on the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-530170 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide an article identification system, a registration system, and an information processing program that can accurately identify an article shown in an image based on the image. [Means for solving the problem]

[0005] An embodiment of the item identification system includes a detection means, an item detection means, a selection means, and an identification means. The key point detection means detects key points of a hand that appear in each of a plurality of images captured by at least one imaging device during a predetermined period. The item detection means detects an item that appears in each of a plurality of images for which the key point detection means is to detect key points. The selection means selects, from among the images in which an item has been detected by the item detection means, an image that has the fewest number of key points detected by the key point detection means. The identification means identifies the item based on an image of a region in which the item detected by the item detection means appears among the images selected by the selection means. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a block diagram showing a schematic configuration of a transaction processing system according to an embodiment and a main circuit configuration of a cart terminal. [Figure 2] FIG. 10 is a perspective view showing an example of a state in which a cart terminal is attached to a cart. [Figure 3] 10 is a flowchart of a registration process. [Figure 4] 10 is a flowchart of a registration process. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system 1 according to this embodiment and a main circuit configuration of a cart terminal 100. As shown in FIG. The transaction processing system 1 is configured so that a cart terminal 100, a payment machine 200, a POS server 300, and an attendant terminal 400 can communicate with each other via a communication network 2.

[0008] The communication network 2 may be the Internet, a VPN (virtual private network), a LAN (local area network), a public communication network, a mobile communication network, or the like, either alone or in appropriate combination. As an example, the communication network 2 may be a combination of the Internet and a mobile communication network. Although any number of cart terminals 100, accounting machines 200, POS servers 300, and attendant terminals 400 may be included in transaction processing system 1, only one of each is shown in FIG.

[0009] The cart terminal 100 is an information processing terminal attached to a shopping cart (hereinafter referred to as a cart) provided in a store. The cart terminal 100 is loaned to a customer who is shopping in the store along with the cart. The cart terminal 100 is a terminal device that, in response to a customer's operation, executes a registration process for registering an item that the customer plans to purchase as a purchased item. As will be described later, the cart terminal 100 executes a process for recognizing an item to be registered as a purchased item in relation to an object shown in an image. In other words, the cart terminal 100 functions as an item identification system. Furthermore, as will be described later, the cart terminal 100 executes a process for registering the recognized item as a purchased item. In other words, the cart terminal 100 functions as a registration system.

[0010] The payment machine 200 is installed in a store and performs payment processing related to the settlement of the price of purchased items registered by the cart terminal 100. The payment machine 200 is operated by an operator during the payment processing. The operator of the payment machine 200 is typically a customer, although a store clerk may also operate the payment machine 200. The POS server 300 is an information processing device that executes management processing for managing the purchase and sale of goods that are processed by the registration processing in the cart terminal 100 and the transaction processing in the transaction device 200 . Attendant terminal 400 is an information processing terminal operated by a store clerk. Attendant terminal 400 is a terminal device for a user interface related to information processing to support the work of the store clerk regarding transactions processed by transaction processing system 1. The work of the store clerk includes, for example, monitoring the implementation status of transactions in progress and providing appropriate support to customers.

[0011] The cart terminal 100 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, a touch panel 104, a sound unit 105, an interface unit 106, a wireless communication unit 107, and a transmission path 108. The processor 101, the main memory unit 102, the auxiliary memory unit 103, the touch panel 104, the interface unit 106, and the wireless communication unit 107 are capable of communicating with each other via the transmission path 108.

[0012] The processor 101, the main storage unit 102, and the auxiliary storage unit 103 are connected via a transmission line 108 to form a computer that performs information processing to realize the functions of the cart terminal 100. The processor 101 corresponds to the central part of the computer and executes information processing in accordance with an operating system and information processing programs such as application programs.

[0013] The main memory unit 102 corresponds to the main memory portion of the computer. The main memory unit 102 includes a read-only memory area and a rewritable memory area. The main memory unit 102 stores part of the information processing program in the read-only memory area. The main memory unit 102 may also store data required for the processor 101 to execute processes for controlling each component in the read-only memory area or the rewritable memory area. The main memory unit 102 uses the rewritable memory area as a work area for the processor 101.

[0014] The auxiliary storage unit 103 corresponds to the auxiliary storage portion of the computer. The auxiliary storage unit 103 may be, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), an SSD (solid state drive), or any other well-known storage device. The auxiliary storage unit 103 stores data used by the processor 101 when performing various processes and data generated by the processes performed by the processor 101. The auxiliary storage unit 103 may also store the information processing program. In this embodiment, the auxiliary storage unit 103 stores a cart terminal program PRA, which is one of the information processing programs. The cart terminal program PRA is an application program that describes the registration process procedures. A portion of the storage area of ​​the auxiliary storage unit 103 is used as an area for storing transaction data DAA. The transaction data DAA is data representing the contents of one transaction.

[0015] The touch panel 104 displays a screen for presenting information to the operator, and also inputs instructions by the operator touching the screen through the touch panel 104. The sound unit 105 outputs sounds for various guidance and warnings. As the sound unit, various well-known sound devices such as a voice synthesis device and a buzzer can be used alone or in combination.

[0016] External devices such as cameras 198 and 199 are connected to interface unit 106. Interface unit 106 acts as an interface for sending and receiving data to and from the connected external devices. An existing USB (universal serial bus) controller or the like can be used as interface unit 106. Cameras 198 and 199 capture images of the process of hand-held products being placed into the basket of a cart from different directions. Cameras 198 and 199 then output image data representing the captured images. Thus, cameras 198 and 199 are both examples of imaging devices. The cameras 198 and 199 may automatically repeat photographing at regular intervals, or may photograph in response to instructions from the processor 101.

[0017] The wireless communication unit 107 executes communication processing for wirelessly performing data communication via the communication network 2. For example, an existing wireless communication device for a wireless LAN can be used as the wireless communication unit 107. Note that instead of or in addition to the wireless communication unit 107, a communication unit that is wired connected to the communication network 2 may be used. The transmission path 108 includes an address bus, a data bus, and control signal lines, and transmits data and control signals exchanged between the connected components. As the basic hardware of the cart terminal 100, it is assumed that the hardware of a tablet-type information processing device, for example, is used.

[0018] FIG. 2 is a perspective view showing an example of how the cart terminal 100 is attached to the cart. The cart 900 includes a caster portion 910 , a handle frame portion 920 , a basket portion 930 , a support portion 940 , and a battery case 950 . The caster unit 910 has four wheels 911 for smoothly moving the cart 900 on the floor surface. The wheels 911 are attached to the frame 912 in a state where they can rotate around an axis in the vertical direction.

[0019] The handle frame portion 920 includes a pair of vertical frames 921, 921 and a handle bar 922. The vertical frames 921, 921 are erected above the two wheels of the caster portion 910. The handle bar 922 connects the upper ends of the vertical frames 921, 921 together. The basket section 930 is provided horizontally from the middle of the handle frame section 920. The basket section 930 is shaped like a basket with an opening that allows products to be placed inside, and functions as a container for storing products that customers plan to purchase. Cameras 198, 199 are attached to the upper end of the basket section 930 on the side away from the handlebar 922 in a manner that allows them to photograph the opening of the basket section 930 from the side.

[0020] The support unit 940 includes a pole 941. The pole 941 is attached to one of the vertical frames 921 so that its tip is positioned higher than the handlebars 922. The cart terminal 100 described above is attached to the tip of the pole 941. As a result, the support unit 940 supports the cart terminal 100 in the state shown in FIG. The battery case 950 is attached between the vertical frames 921, 921 on the lower end side of the handle frame portion 920. The battery case 950 houses a battery used as an external power source for the cart terminal 100.

[0021] Next, the operation of the transaction processing system 1 configured as described above will be described. Note that the contents of the various processes described below are merely examples, and it is possible to change the order of some of the processes, omit some of the processes, or add other processes as appropriate. For example, in the following explanation, in order to clearly explain the characteristic operations of this embodiment, explanation of some of the processes will be omitted. For example, if some kind of error occurs, processing may be performed to deal with the error, but a description of such processing will be omitted.

[0022] The customer performs a predetermined operation to start using the unused cart terminal 100. When this operation is performed, the processor 101 starts a registration process based on the cart terminal program PRA. 3 and 4 are flowcharts of the registration process.

[0023] 3, the processor 101 generates new transaction data DAA for the target transaction. That is, the processor 101 determines a new transaction code that is different from transaction codes for identifying other transactions in accordance with a predetermined rule, generates new transaction data DAA that includes this transaction code but does not include product information about the purchased product, and stores the new transaction data DAA in the auxiliary storage unit 103.

[0024] In ACT12, the processor 101 checks whether it is time to acquire an image. If the processor 101 cannot confirm the event, it determines NO and proceeds to ACT13. The acquisition timing is assumed to be, for example, the timing immediately after proceeding from ACT11 to ACT12, and the timing at regular time intervals thereafter. However, another timing may be determined as the acquisition timing, and is determined as appropriate, for example, by the creator of the cart terminal program PRA. However, the acquisition timing should be determined so that an image arrives at least once while a commodity being put into the basket section 930 passes through the imaging range of the cameras 198, 199. In ACT13, the processor 101 checks whether a payment has been requested. If the processor 101 cannot confirm this event, it determines "NO" and returns to ACT12. Thus, in ACT12 and ACT13, the processor 101 waits for the acquisition timing to occur or for an accounting request to be made.

[0025] When the acquisition timing arrives, the processor 101 determines YES in ACT 12 and proceeds to ACT 21 and ACT 31. In other words, when the processor 101 determines YES in ACT 12, it transitions to a state in which it processes the processing routine starting from ACT 21 and the processing routine starting from ACT 31 in parallel. However, the processor 101 may also execute these processing routines sequentially in a time-division manner.

[0026] In ACT21, processor 101 acquires image data (hereinafter referred to as first image data) output by camera 198. For example, if camera 198 repeatedly captures images at regular intervals, processor 101 acquires first image data output by camera 198 in response to the most recent image capture. For example, if camera 198 captures images in response to an instruction from processor 101, processor 101 instructs camera 198 to capture images, and acquires the first image data output by camera 198 in response to this instruction. Then, processor 101 stores the acquired first image data in main storage unit 102 or auxiliary storage unit 103.

[0027] In ACT22, the processor 101 detects the first number of key points. For example, the processor 101 attempts to extract an area in which a hand is reflected in an image represented by the first image data (hereinafter referred to as the first image). If the processor 101 successfully extracts the area, it determines each of the key points of the reflected hand. In this embodiment, the key points are the finger joints, fingertips, and wrist. However, the key points may be appropriately determined by, for example, the creator of the cart terminal program PRA, such as by limiting the key points to the joints. The processor 101 then determines the number of determined key points as the first number of key points. If no hand is reflected in the first image, the processor 101 sets the first number of key points to 0. Alternatively, if a hand is reflected in the first image but no key points can be determined, the processor 101 sets the first number of key points to 0. The keypoints here can be determined by any known method, such as deep learning using a convolutional neural network (CNN), which is a method suitable for image recognition. Thus, by having the processor 101 execute information processing based on the cart terminal program PRA, the computer with the processor 101 as its central part functions as a keypoint detection means.

[0028] In ACT23, the processor 101 detects a first product area. For example, the processor 101 attempts to extract an area in which a product is reflected in the first image. If the processor 101 successfully extracts the area, the processor 101 designates the area as the first product area. The area to be extracted can be determined using well-known methods, such as deep learning using CNN or graph cuts, and the method is not critical. In this embodiment, the product to be purchased is an item that can be held by a customer. In other words, when detecting the first product area, the processor 101 detects the product as an item reflected in the first image. Thus, the processor 101 executes information processing based on the cart terminal program PRA, and the computer including the processor 101 as its central part functions as an item detection means.

[0029] In ACT31, ACT32, and ACT33, processor 101 performs the same processing as in ACT21 to ACT33 on image data (hereinafter referred to as second image data) output by camera 199. As a result, processor 101 determines the number of key points reflected in the image represented by the second image data (hereinafter referred to as second image) as the second key point count. Processor 101 also attempts to extract a second product area as an area in which a product is reflected in the second image.

[0030] A customer searches for a product to purchase in a store. The customer then removes the product from the sales floor and places it in the basket 930. At this time, the hand and the product pass through the shooting range of cameras 198, 199 and are captured in the first and second images. However, the number of key points captured in each of the first and second images varies depending on the orientation of the hand and the relative positions of the hand and the product. There are cases where no key points are captured in either or both of the first and second images. There are also cases where the product is not captured in either or both of the first and second images. If an image in which the hand and the product are captured is acquired as the first or second image, the number of first or second key points is detected, and the first or second product area is extracted.

[0031] After completing both ACT23 and ACT33, the processor 101 proceeds to ACT41 in FIG. In ACT 41, the processor 101 checks whether a key point has been identified in either the first image or the second image. If at least one key point has been identified in ACT 22 or ACT 32 in Figure 3, the processor 101 determines YES and proceeds to ACT 42 in Figure 4.

[0032] In ACT42, the processor 101 checks whether the first product area or the second product area has been extracted. If the first product area or the second product area has been extracted in ACT23 or ACT33 in Figure 3, the processor 101 determines the answer as YES and proceeds to ACT43 in Figure 4. In ACT43, the processor 101 selects either the first image or the second image as the image to be subjected to the commodity recognition process (hereinafter referred to as the target image) based on the number of first key points and the number of second key points detected in ACT22 and ACT32 in FIG. 3. For example, if the number of first key points is smaller than the number of second key points, the processor 101 selects the first image as the target image. For example, if the number of second key points is smaller than the number of first key points, the processor 101 selects the second image as the target image. Thus, by the processor 101 executing information processing based on the cart terminal program PRA, the computer with the processor 101 as its central part functions as a selection means.

[0033] For example, if the number of first key points is the same as the number of second key points, the processor 101 selects either the first image or the second image as the target image according to a predetermined rule. The selection rule here may be determined as appropriate by, for example, the creator of the cart terminal program PRA. As an example, the processor 101 selects the first image if the size of the first product area extracted in ACT23 in FIG. 3 is larger than the size of the second product area extracted in ACT33, and selects the second image if the size of the second product area extracted in ACT33 in FIG. 3 is larger than the size of the first product area extracted in ACT32. Alternatively, the processor 101 may select a predetermined one of the first image and the second image. Alternatively, the processor 101 may randomly select one of the first image and the second image.

[0034] If the processor 101 cannot determine a key point in either the first image or the second image, it determines NO in ACT41 in FIG. 4 and proceeds to ACT44. In ACT 44, the processor 101 checks whether the first product area or the second product area has been extracted. If the first product area or the second product area has been extracted in ACT 23 or ACT 33 in Figure 3, the processor 101 determines the answer as YES and proceeds to ACT 45 in Figure 4.

[0035] In ACT45, the processor 101 selects either the first image or the second image as the target image based on the extraction status of the first product area or the second product area. For example, if the size of the first product area extracted in ACT23 in FIG. 3 is larger than the size of the second product area extracted in ACT33, the processor 101 selects the first image, and if the size of the second product area extracted in ACT33 in FIG. 3 is larger than the size of the first product area extracted in ACT32, the processor 101 selects the second image. The rules for selecting the target image here may be determined as appropriate, for example, by the creator of the cart terminal program PRA. For example, the processor 101 may select a predetermined one of the first image and the second image. Alternatively, the processor 101 may randomly select one of the first image and the second image.

[0036] Once the processor 101 has finished selecting the target image in ACT43 or ACT45, it proceeds to ACT46 in either case. As ACT46, the processor 101 executes an identification process. The identification process is a process of identifying a product reflected in a target image based on the target image. The method of this identification process may be, for example, deep learning using CNN, which is a method suitable for image recognition. Alternatively, any method may be used, such as OCR (optical character recognition) or feature point matching. Thus, by the processor 101 executing information processing based on the cart terminal program PRA, the computer with the processor 101 as its central part functions as an identification means.

[0037] In ACT 47, the processor 101 checks whether the identification in ACT 46 was successful. If the identification was successful, the processor 101 determines YES and proceeds to ACT 48. In ACT48, the processor 101 checks whether a product that is a candidate for registration as a purchased product (hereinafter referred to as a candidate product) has already been set. If the processor 101 cannot confirm the event, it determines NO and proceeds to ACT49. In ACT 49, the processor 101 sets the product identified in ACT 46 as a candidate product. Then, the processor 101 returns to the standby state in ACT 12 and ACT 13 in FIG.

[0038] If the processor 101 is unable to identify the product in ACT46, it determines NO in ACT47, skips ACT48 and ACT49, and returns to the standby state of ACT12 and ACT13 in Fig. 3. Even if the processor 101 is successful in identifying the product in ACT46, if a candidate product has already been set, it determines YES in ACT48, skips ACT49, and returns to the standby state of ACT12 and ACT13 in Fig. 3.

[0039] Thus, while the customer is trying to place the product he or she wishes to purchase into the basket section 930 and the hand and product pass through the range of the cameras 198, 199, the processor 101 repeats ACT21 to ACT23, ACT31 to ACT33, and ACT41 to ACT49 each time an acquisition timing occurs, and the product that is first identified in ACT46 is set as the candidate product.

[0040] Then, when the items have been placed in the basket section 930 and the hand and items are no longer reflected in the first image or the second image, it becomes impossible to determine key points from either the first image or the second image, and it becomes impossible to extract either the first item region or the second item region. In response to this, the processor 101 determines NO in both ACT41 and ACT44 in Figure 4, and proceeds to ACT50.

[0041] In ACT50, the processor 101 checks whether there are any candidate products. If candidate products have already been set in ACT49 as described above, the processor 101 determines YES and proceeds to ACT51. In ACT51, the processor 101 checks whether a predetermined determination condition for determining a candidate product as a purchased product is met. The determination condition may be determined as appropriate, for example, by the creator of the cart terminal program PRA. One example of the determination condition is assumed to be "YES is determined in ACT50 at each of a predetermined number of consecutive acquisition timings." If the determination condition is not met, the processor 101 determines NO and returns to the standby state of ACT12 and ACT13 in FIG. 3. Thus, even if the candidate product has been set and the hand and product are not captured in the first image or the second image, the processor 101 repeatedly executes the processes of ACT21 to ACT23 and ACT31 to ACT33 at each acquisition timing until the determination condition is met. If the determination condition is met, the processor 101 determines YES in ACT51 in FIG. 4 and proceeds to ACT52.

[0042] In ACT52, the processor 101 registers the candidate product as a purchased product. That is, the processor 101 updates the transaction data to include the product code of the candidate product as the product code of the purchased product, for example. The processor 101 then returns to the standby state of ACT12 and ACT13 in Figure 3. In this way, the processor 101 executes information processing based on the cart terminal program PRA, and the computer with the processor 101 as its central part functions as registration means.

[0043] The processor 101 cancels the setting of a product as a candidate product once it has been registered as a purchased product. Therefore, until a new hand and product appear in the first image and the second image, the processor 101 judges NO in both ACT41 and ACT44 and proceeds to ACT50. However, since it is not possible to confirm in ACT50 that a candidate product exists, the processor 101 judges NO, skips ACT51 and ACT52, and returns to the standby state of ACT12 and ACT13 in FIG. 3.

[0044] When a customer attempts to place a product held in his / her hand into the basket section 930, and the hand is reflected in at least one of the first image and the second image, even if a key point can be determined in ACT22 or ACT32 in Figure 3, it may not be possible to extract the first product area or the second product area in ACT23 or ACT33.

[0045] In such a case, the processor 101 determines YES in ACT 41 and NO in ACT 42 in FIG. 4, and proceeds to ACT 53. As ACT53, the processor 101 executes a predetermined alert action. The alert action is an action to notify the customer that the product may be held improperly. For example, the processor 101 controls the touch panel 104 to display a guidance screen instructing the customer to hold the product so that it will be captured by the camera. Alternatively, the processor 101 controls the sound unit 105 to output a voice message instructing the customer to hold the product so that it will be captured by the camera. The alert action may be any of a variety of actions that the customer can perceive, such as a flashing screen on the touch panel 104 or an audible alarm from the sound unit 105. The processor 101 may also execute multiple actions in parallel as the alert action. The action to be executed as the alert action may be determined as appropriate by, for example, the creator of the cart terminal program PRA. Alternatively, the action to be executed as the alert action may be determined as appropriate by, for example, the administrator of the cart terminal 100. The processor 101 then returns to the standby state of ACT12 and ACT13 in FIG. 3 . Thus, by the processor 101 executing information processing based on the cart terminal program PRA, the computer with the processor 101 as its central part realizes the function as an informing means in cooperation with the touch panel 104 or the sound unit 105, etc.

[0046] After registering the purchased items, the customer performs a predetermined operation, for example, on the touch panel 104, to indicate the start of the transaction. In response, the processor 101 determines YES in ACT13 in FIG. 3 and proceeds to the transaction process. The transaction process may be similar to that performed in existing cart POS systems, and is not shown in the figures. For example, the processor 101 performs the transaction process by transferring transaction data to the transaction device 200. Alternatively, if a payment terminal (not shown) is attached to the cart 900 and the payment terminal is connected to the cart terminal 100, the transaction process may be the process by which credit card payments, code payments, etc. are made using the payment terminal.

[0047] As described above, the cart terminal 100 selects the image from the first image and the second image that has the fewer number of key points detected for the reflected hand as the target image, and identifies the product reflected in that target image based on that target image. As a result, of the first image and the second image in which the positional relationship between the hand and the product is different from each other, the image that is not hidden by the hand and that better reflects the external features of the product is used as the target image for product identification, and the product reflected in the image can be accurately identified based on that image.

[0048] Furthermore, the cart terminal 100 uses, as the first image and the second image, images captured at the same time by two cameras 198, 199 with different shooting directions. This makes it highly likely that the positional relationship between the reflected hand and the product will be significantly different between the first image and the second image, increasing the possibility of selecting an image that better captures the external features of the product as the target image.

[0049] Furthermore, if the cart terminal 100 can detect key points from either the first image or the second image but cannot extract either the first product area or the second product area, it will issue an alert to notify the customer of the abnormality, thereby making the customer aware that they are holding the product improperly.

[0050] Furthermore, the cart terminal 100 registers the identified product as a purchased product, so that the customer can register the purchased product simply by placing the product he or she is holding in his or her hand into the basket section 930.

[0051] This embodiment can be modified in various ways as follows. Three or more images obtained by three or more cameras may be included as candidates for the target image.

[0052] A plurality of images obtained at different times by a single camera may be included in the candidates for the target image.

[0053] If the processor 101 determines YES in ACT42 in FIG. 4, it may perform classification processing on each of the first image and the second image as target images. The product may then be identified based on the results of these two classification processes. As an example, if a certainty is obtained as a result of the classification process, it is possible to adopt the classification result with the greater certainty. Alternatively, the product may be identified by integrating the results of multiple classification processes, such as identifying the product based on the average value of the certainty obtained for the same product as a result of the classification processes.

[0054] It is also possible to recognize items other than merchandise.

[0055] The results of the recognition of the product may be used for any purpose, not just for purchase registration for transaction processing.

[0056] In the above embodiment, part of the processing performed by processor 101 may be executed by a computer provided in POS server 300 or any one or more other information processing devices. For example, cart terminal 100 may perform only the user interface operation, and various information processing for identifying the purchased item may be executed by a computer provided in POS server 300. In other words, each of the means for controlling devices to realize the functions of the key point detection means, item detection means, selection means, identification means, registration means, or notification means may be provided in any information processing device other than cart terminal 100 included in transaction processing system 1, or may be appropriately distributed among multiple information processing devices included in transaction processing system 1.

[0057] Some or all of the functions realized by the processor 101 through information processing can be realized by hardware that executes information processing not based on a program, such as a logic circuit, etc. Each of the above functions can also be realized by combining hardware such as the above logic circuit with software control.

[0058] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0059] 1...transaction processing system, 2...communication network, 100...cart terminal, 101...processor, 102...main memory unit, 103...auxiliary memory unit, 104...touch panel, 105...sound unit, 106...interface unit, 107...wireless communication unit, 108...transmission path, 198...camera, 199...camera, 200...accounting machine, 300...POS server, 400...attendant terminal, 900...cart.

Claims

1. a key point detection means for detecting a key point of a hand appearing in each of a plurality of images captured by at least one photographing device during a predetermined period; an object detection means for detecting an object reflected in each of a plurality of images for which the key point detection means detects key points; a selection means for selecting an image having a smaller number of key points detected by the key point detection means from among the images in which an object has been detected by the object detection means; an identification means for identifying the object based on an image of a region in which the object detected by the object detection means is reflected in the image selected by the selection means; An article identification system comprising:

2. the keypoint detection means detects keypoints for each of a plurality of images simultaneously captured by a plurality of image capture devices that capture images of the same area from different directions; the item detection means detects an item in each of the same images as those from which the keypoint detection means detects keypoints; The item identification system of claim 1 .

3. an alarm means for performing an alarm operation to alarm an abnormality when a key point is detected from any of the plurality of images by the key point detection means and an article is not detected from any of the plurality of images by the article detection means; The item identification system of claim 1 further comprising:

4. the identification means identifies the article based on the appearance characteristics of the article that appear in the image of the region in which the article detected by the article detection means is reflected, among the images selected by the selection means; The item identification system of claim 1 .

5. At least one photographing device that photographs a product being held by hand and placed into a container containing the product to be purchased so as to capture the product; a key point detection means for detecting a key point of a hand appearing in each of a plurality of images captured by the photographing device during a predetermined period; an object detection means for detecting an object reflected in each of a plurality of images for which the key point detection means detects key points; a selection means for selecting an image having a smaller number of key points detected by the key point detection means from among the images in which an object has been detected by the object detection means; an identification means for identifying which product the item is based on an image of a region in the image selected by the selection means in which the item detected by the item detection means is reflected; a registration means for registering the commodity identified by the identification means as a commodity to be purchased; A registration system equipped with

6. Computer, a key point detection means for detecting a key point of a hand appearing in each of a plurality of images captured by at least one photographing device during a predetermined period; an object detection means for detecting an object reflected in each of a plurality of images for which the key point detection means detects key points; a selection means for selecting an image having a smaller number of key points detected by the key point detection means from among the images in which an object has been detected by the object detection means; an identification means for identifying the object based on an image of a region in which the object detected by the object detection means is reflected in the image selected by the selection means; An information processing program that makes it function as such.

Images