Information processing device, information processing system, and computer program

Abstract

To provide a device that can easily register an image for learning used in object scanning.SOLUTION: A measuring device 1 comprises: an imaging unit 111 that picks up an image of a commodity; an extraction unit 125 that refers to a commodity file 121 storing commodity information on the basis of the image of the commodity picked up by the imaging unit 111, and extracts candidates for a commodity to which an image for learning is added; a designation receiving unit 131 that receives designation of a commodity in which the image for learning is additionally registered from the candidates for the commodity extracted by the extraction unit 125; and a registration unit 127 that registers, in the commodity file 121, the image of the commodity picked up by the imaging unit 111 as the image for learning in association with the commodity designated in the designation receiving unit 131.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, and a computer program.

Background Art

[0002] A so-called object scan technology is known in which an imaging image of an article obtained by imaging the article is collated with a learning image as a reference image for image recognition to identify the article.

[0003] For example, Patent Document 1 discloses an article identification apparatus including a tray on which an article is placed, a backlight light source that illuminates backlight light from the bottom surface side of the tray, a digital camera that images the article on the tray, and an identification computer that performs image recognition on an image from the digital camera and identifies the article. This article identification apparatus cuts out an article on the tray from a difference image between an image when the backlight light source is on and an image when the backlight light source is off.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In this regard, in image learning for object scan, an article is imaged for each target product, and an image is registered by designating the article each time, which is very troublesome.

[0006] Therefore, one object of the present invention is to provide an apparatus capable of easily registering a learning image used for object scan.

Means for Solving the Problems

[0007] To achieve the above objective, the information processing device according to the present invention comprises: an imaging means for capturing images of products; an extraction means for extracting candidate products to which learning images are to be added, by referring to a product information storage means that stores product information based on the images of products captured by the imaging means; a designation receiving means for receiving designations of products to which the learning images are to be added from the candidate products extracted by the extraction means; and a registration means for registering the images of products captured by the imaging means as learning images in the product information storage means in association with the products designated by the designation receiving means. [Brief explanation of the drawing]

[0008] [Figure 1] This is a configuration diagram showing the configuration of a system equipped with a weighing device according to an embodiment of the present invention. [Figure 2] This is a functional block diagram showing the hardware configuration of a weighing device according to an embodiment of the present invention. [Figure 3] This is a functional block diagram showing the software configuration of a weighing device according to an embodiment of the present invention. [Figure 4] This is an example of a screen displayed on a weighing device according to an embodiment of the present invention, showing a standby screen in the first operating mode. [Figure 5] An example of a screen displayed on a weighing device according to an embodiment of the present invention, showing a screen for specifying a product in the first operating mode. [Figure 6] An example of a screen displayed on a weighing device according to an embodiment of the present invention, showing a screen for specifying a product in a second operating mode. [Figure 7] An example of a screen displayed on a weighing device according to an embodiment of the present invention, showing a screen for specifying a product in the third operating mode. [Figure 8] An example of a screen displayed on a weighing device according to an embodiment of the present invention, showing a screen for specifying a product in the fourth operating mode. [Figure 9] This is a processing flow diagram showing an example of processing by a weighing device according to an embodiment of the present invention, illustrating the entire process in the first operating mode. [Figure 10] This is a processing flow diagram illustrating an example of processing by a weighing device according to an embodiment of the present invention, specifically showing the process of extracting the "product" item from the processing in the first operating mode. [Figure 11] This is a processing flow diagram showing an example of processing by a weighing device according to an embodiment of the present invention, illustrating the entire process in the second operating mode. [Figure 12] This is a processing flow diagram showing an example of processing by a weighing device according to an embodiment of the present invention, illustrating the entire process in the third operating mode. [Figure 13] This is a processing flow diagram showing an example of processing by a weighing device according to an embodiment of the present invention, illustrating the entire process in the fourth operating mode. [Modes for carrying out the invention]

[0009] ● System Configuration Hereinafter, an information processing apparatus according to an embodiment of the present invention will be described with reference to the figures. In the following explanation, unless otherwise specified, functional units and processes that are assigned the same codes as those previously described will be assumed to be configured in the same way as previously described functional units and processes, and their explanation will be omitted.

[0010] Figure 1 shows an example of the configuration of a system equipped with the weighing device 1 according to this embodiment, and Figure 2 shows the hardware configuration of the weighing device 1. The weighing device 1 according to this embodiment is one embodiment of the information processing device according to the present invention. The weighing device 1 is installed, for example, in a supermarket that sells fresh foods such as vegetables and fruits. In the fresh food section of a supermarket, products such as vegetables and fruits that are sold individually are sorted by type and displayed in their respective display areas. Since each item is sold individually, it does not have a label with the price or barcode printed on it. The customer or store clerk weighs the required amount of each type of item and receives labels for the items to be purchased from weighing device 1. The customer or store clerk then attaches the labels to the items. The labels have a symbol code printed on them that contains information such as the price. By having the symbol code read by POS terminal 3, the items can be registered as items to be purchased at the POS terminal 3.

[0011] The weighing device 1 displays a list of selectable products on the screen of the display touch panel 104 and accepts product selections from the operator, such as a customer or store clerk. The weighing device 1 also weighs the product placed on the weighing unit 109 and retrieves product data, such as the unit price of the product specified by the operator on the display touch panel 104, from a storage unit such as RAM 103, calculates the price of the product, and issues a label. In particular, the weighing device 1 according to this embodiment performs an object scan of the product selected by the customer or store clerk and presents the candidate product to the customer or store clerk so that they can select it. Here, "object scan" means the process of capturing an image of the product, performing image recognition of the product, and identifying the product by comparing it with a training image, and is performed by the imaging control unit 124 and the extraction unit 125 (see Figure 3), which will be described later.

[0012] Furthermore, some or all of the product data, including the product file 121 (see Figure 3) described later, may be stored not only in the memory unit of the weighing device 1, such as RAM 103, but also in an appropriate configuration connected by wire or wireless. In particular, the training images and training data for object scanning, described later, do not need to be stored in the weighing device 1 itself, but may be stored, for example, in a higher-level store controller 2 or cloud server 4, and the weighing device 1 may refer to them.

[0013] The store controller 2 is a computer that manages various types of data within a supermarket and is connected to the weighing device 1, the POS terminal 3, and a LAN (Local Area Network). Examples of the various types of data include a product file that stores information such as the selling price of products stored in the POS terminal 3, a product file 121 (see FIG. 3) stored in the weighing device 1, which will be described later, and a performance file transmitted from the POS terminal 3, etc.

[0014] The POS terminal 3 is a so-called POS register that performs product registration, payment settlement, etc. for products purchased by customers (including fresh food weighed by the weighing device 1 and products with labels pre-attached). On the product label, product identification numbers such as product codes, and in the case of labels issued by the weighing device 1, price information corresponding to the weight, etc. are printed in the form of barcodes or two-dimensional codes, and the POS terminal 3 is equipped with a reading device that reads these barcodes or two-dimensional codes. Then, the POS terminal 3 performs a product registration process based on the information of the barcodes or two-dimensional codes read by the reading device and the information such as the selling price of the product stored in the memory. Regardless of the illustration, a plurality of POS terminals 3 may be provided within the store. Also, as the POS terminal 3, in addition to those operated by store employees, there may be a self-service terminal operated by customers themselves.

[0015] The cloud server 4 is a device that executes the processes executed by the devices within the store including the weighing device 1 and holds the data necessary for the processes, and is provided as appropriate. In this embodiment, object scanning of products is performed in the weighing device 1, but the weighing device 1 may only perform imaging of products, and the processes related to image recognition and analysis may be executed by the cloud server 4. Also, in the cloud server 4, it may be configured such that deep learning using AI (Artificial Intelligence) generates data for verification (learning images), and the accuracy of object scanning is improved.

[0016] The store controller 2 and cloud server 4 constitute a higher-level management device for the weighing device 1 and POS terminal 3. Regardless of this embodiment, the higher-level device may be composed of a single terminal or device, or it may be composed of a terminal or device different from the cloud server 4. Furthermore, the higher-level device may be a terminal or device inside the store connected to a network such as a LAN, or it may be a terminal or device outside the store connected to a network such as the Internet.

[0017] ● Hardware configuration of weighing device 1 Figure 2 shows the hardware configuration of weighing device 1, and Figure 3 shows the software configuration of weighing device 1. As shown in Figure 2, the weighing device 1 consists of a CPU 101, ROM 102, RAM 103, display touch panel 104, key operation unit 105, clock unit 106, voice input unit 107, communication unit 108, weighing unit 109, printing unit 110, imaging unit 111, and external storage control unit 112 in its hardware configuration.

[0018] The CPU 101 is a central processing unit that controls the operation of the weighing device 1. It reads and executes a program from the ROM 102, thereby enabling the weighing device 1 to operate according to that program. The ROM 102 is a read-only memory in which the above program is stored. The RAM 103 is a write-and-read memory in which the product file 121, icon table, and other temporary data described later are stored.

[0019] The display touch panel 104 is a user input / output interface between the weighing device 1 and the operator, such as a customer. It displays a preset key G221 (see Figure 5, etc.) for specifying the product to be weighed, as well as various other information, and accepts input from the operator for the displayed preset key G221. An example of the display on the display touch panel 104 will be described later. In addition, the product data corresponding to the preset key G221 specified by the operator on the screen is read from the product file 121 in RAM 103, and the price of that product is calculated.

[0020] The key operation unit 105 is for performing inputs other than those performed on the display touch panel 104.

[0021] The clock unit 106 outputs time information. This time information is used to control the display mode of the preset key G221 (see Figure 5, etc.) displayed on the display touch panel 104.

[0022] The voice input unit 107 is for voice input, allowing the operator to input product information by voice.

[0023] The communication unit 108 is a LAN interface and controls communication with the store controller 2. The product file 121 is obtained from the store controller 2 via this communication unit 108. A program may also be transferred from the store controller 2 to the weighing device 1 via the communication unit 108. The transferred program can be stored, for example, in the RAM 103.

[0024] The weighing unit 109 measures the weight of the product placed on the weighing pan located on top of the weighing device 1 using a load cell-based mass detection method, and outputs the measured value. In detail, the weighing unit 109 consists of a mass detection unit and an A / D converter, and the A / D converter converts the analog signal input from the mass detection unit into a digital signal.

[0025] The printing unit 110 is a printing unit that issues labels with the price, barcode, etc., printed on them for products weighed by the weighing unit 109. The labels are printed based on the weighed value output as the weighing result from the weighing unit 109 and product information (such as unit price data) about the product stored in the product file in RAM 103. In this embodiment, the label is a sticker and can be affixed to the product or the bag containing the product. In this embodiment, the information necessary for product settlement is printed on the label and issued, but the invention is not limited to this, and the information may also be printed on the bag containing the product.

[0026] The imaging unit 111 is a device that images the product placed on the weighing unit 109, and is composed of an image sensor such as a color CCD image sensor or a color CMOS image sensor. The number of imaging units 111 is arbitrary, and multiple units may be provided. The imaging units 111 may be arranged facing different angles to each other, so that the item placed on the weighing unit 109 can be imaged from multiple angles.

[0027] A portable storage medium 113, such as a DVD-ROM or USB memory stick, is connected to the external storage control unit 112. The portable storage medium 113 stores information including programs, and the external storage control unit 112 reads and writes this information.

[0028] ● Software configuration of weighing device 1 As shown in Figure 3, in its software configuration, the weighing device 1 uses the aforementioned CPU 101 and other arithmetic units and RAM 103 to configure the following functional blocks: product file 121, weighing control unit 122, change detection unit 123, imaging control unit 124, extraction unit 125, setting unit 126, registration unit 127, grouping unit 128, calculation unit 129, printing control unit 130, designation acceptance unit 131, and display control unit 132. Some or all of the functional units may be implemented on a higher-level device such as a store controller 2 connected by wired or wireless connection, or on a cloud server 4 connected via the Internet or the like.

[0029] Here, the weighing device 1 is mainly switched to the first to fourth operating modes and performs functions corresponding to each operating mode. The first operating mode is used when providing services to customers, while the second to fourth operating modes are so-called maintenance modes for setting product information, and the screen displayed in these maintenance modes can be said to constitute a maintenance screen.

[0030] The first operation mode is an operation mode in which a customer performs operations to purchase or trade goods. In this embodiment, this first operation mode is an operation mode for weighing and pricing goods, in which goods placed on the weighing unit 109 as goods to be purchased are weighed and a label displaying the price is issued. In other embodiments, if the weighing device 1 is equipped with the functions of a POS terminal 3, or if the information processing device according to the present invention is configured as a POS terminal 3, then in this first operation mode, product registration is performed to register the goods as goods to be purchased along with the weighing of the goods. Furthermore, although the weighing device 1 in this embodiment performs weighing and pricing, it may be configured as a device that only performs weighing, and pricing may be performed by another device that cooperates with it.

[0031] The second operating mode is a mode for registering product training images. In this second operating mode, images of products captured by the imaging unit 111 are registered as product training images.

[0032] The third operating mode is for setting products as either products for sale or products not for sale. In this third mode, it is possible to set whether a store will treat products registered in product file 121 as products for sale or products not for sale. Such settings are necessary depending on the products that are offered seasonally and the availability of inventory, as will be described later.

[0033] The fourth operating mode is a mode for grouping products. In this fourth operating mode, multiple types of products that share a common learning image are registered as a single group. In the following descriptions of each function, functions that do not specifically mention a particular mode are performed in common across all modes.

[0034] <Product File 121> The product file 121 is a file that stores product information for each product to be weighed by the weighing device 1, consisting of the following items that constitute product identification information: product code, product name, classification, preset key name, preset key image, weight (mass), unit price, product color, learning image, group information, and sales target setting, and constitutes the product information storage means. In addition, the product file 121 can also register barcode classification (in-store barcode system), cost, unit weight, tare weight, etc. Furthermore, this product file 121 may be created by the store controller 2 or cloud server 4 and transmitted to the weighing device 1 for storage in RAM 103. However, this does not prevent the weighing device 1 from creating the product file 121 in other embodiments. Also, the selling price may be updated at a predetermined time, such as once a day. The barcode classification system defines the classifications related to PLU codes such as JAN (EAN) codes, and the barcode system (types) used when in-store barcodes are used, such as for bulk-sold goods.

[0035] A product code is product identification information and is also called a product number. The product name is the name of the product and is set for search purposes. The classification is a category (item) that corresponds to the nature of each product. By classifying and registering each product in a predetermined category, when a designated category tab is selected, the products belonging to that category will be filtered. The preset key name is the product name or other information displayed for preset key G221 (see Figure 5, etc.). If left blank, the general name of the product will be used. The preset key image is used to display the preset key G221, which is used to accept product selection. The preset key image represents a design, photograph, or text information that allows the product to be identified.

[0036] The weight data is set as the average value per item and is used to calculate the number of items from the weighing values ​​measured by the weighing device 1. This weight data can also be used to determine if multiple items are placed on the weighing unit 109. For this reason, this data is set only for items sold by the number of items, such as apples. For items not sold by the number of items, such as meat sold by the gram, the price is calculated using only the unit price data (price per gram). For items sold by the number of items, the unit price data is set as the price per item, and for items sold by the gram, it is set as the price per 100 grams, for example. In addition, to allow apples and other items to be sold by the gram, it may be possible to set "sold by the gram" or "sold by the number" in the sales category. In the case of "sold by the number of items," the transaction amount is calculated according to the set unit weight. The unit weight may be registered by inputting the number of items placed or by the measured value. Also, when multiple items are weighed, the number of items placed on the weighing plate may be entered, and the average weight calculated by division based on that number may be stored. The product color is specified as the color associated with the product. To accommodate various perceptions of product color, one or more colors can be arbitrarily registered.

[0037] The training images are images of products, and they serve as reference images used when recognizing and identifying products in the captured images through object scanning, and when extracting product candidates from the product file 121. Training images can be registered by capturing images of actual products with the imaging unit 111, or they may be registered by reading pre-captured image data. For example, a product code can be specified, and training images of the target product can be captured and registered sequentially. In addition, multiple training images can be registered for a single product. Furthermore, new training images may be automatically generated and registered from already registered training images using AI-based deep learning.

[0038] Group information is information that defines a group of products to which images of products captured by the imaging unit 111 are associated as training images. That is, for products set in a predetermined group, the training image of one product belonging to that predetermined group is also associated with other products belonging to that predetermined group. For example, if there are "apples" which are the same product in terms of fruit variety, but are sold from different regions or brands, when these are grouped, each "apple" from each region or brand is treated as belonging to the same group regardless of the differences in region or brand. Then, as described later, when a training image is associated with one product belonging to the same group, the same training image is also associated with other products belonging to the same group.

[0039] The sales target setting is information indicating whether a store will handle a product as a sales target or as an ineligible product, and is set by the setting unit 126, which will be described later. Some of the products handled by the weighing device 1, i.e., the products registered in the product file 121, are seasonal or not stocked year-round. Therefore, it is not appropriate to display products that are not currently for sale on the screen during weighing, and excluding them from the object scan improves the accuracy of the object scan. This sales target setting can be done for each product by calling up a predetermined product by part number from the product list, but in this embodiment, it can be done by specifying a predetermined product from the product candidates extracted by the extraction processing unit using the setting unit 126, which will be described later.

[0040] Furthermore, the setting of eligible products for sale is not limited to cases where the operator specifies a product to be sold or not sold; it may also be possible to set it so that products are automatically treated as eligible or ineligible for sale according to a pre-set period or the like. For example, similar to setting a period for a special sale in a product promotion, a sales period may be set for a product, such as from one specified date to another, and the product may be treated as eligible or ineligible for sale depending on whether the date and time of use of the weighing device 1 falls within the sales period. That is, a sales period is set for a product in the product file 121, and when the weighing device 1 is powered on, it acquires calendar information held by itself or any external terminal. Referring to the calendar information, it determines whether the current date and time for a specified product falls within the sales period, and if it does, it treats the specified product as eligible for sale; otherwise, it treats the specified product as ineligible for sale.

[0041] Furthermore, the sales period can also be set according to the season. For example, in product file 121, the season in which a product is treated as a product for sale is set, and a separate season master is created that associates dates with seasons. The weighing device 1 refers to the season master to determine the season for the day and sets the product as either a product for sale or a product not for sale.

[0042] Here, the season set for a product is not limited to just one of the four seasons (spring, summer, autumn, or winter); overlapping seasons, such as spring / summer and autumn / winter, are also possible. In this case, the product may be treated as a saleable item as long as it falls within one of the overlapping seasons, or the operator may be allowed to select from a pop-up screen which season the product should be sold or not sold in. Furthermore, the season master may include a link between the location information of the store where the weighing device 1 is installed and the season. This allows the season setting to be reflected according to location, such as spring in Hokkaido and summer in Okinawa, even at the same time and date. Furthermore, the seasonal sales target setting may be toggled ON / OFF by an arbitrary operation such as a designated button, allowing the user to decide whether or not to reflect the seasonal sales target setting for the product on that day. In addition, the seasonal master may be categorized in product file 121 and referenced from there.

[0043] Furthermore, the sales period can also be set on a monthly basis. For example, in product file 121, a flag can be set for each product to indicate whether or not it should be treated as a product for sale during a given month. The weighing device 1 determines the month of the current day based on the date of the current day and sets the product as a product for sale or as a product for sale according to the flag.

[0044] Furthermore, regarding the setting of the sales period, both seasonal and monthly settings may be provided, allowing the operator to selectively set whichever is more convenient.

[0045] <Weighing control unit 122> The weighing control unit 122 controls the weighing unit 109 to weigh the product placed on the weighing pan of the weighing unit 109.

[0046] <Change detection unit 123> The change detection unit 123 is a functional unit that detects when the weight on the weighing pan being weighed by the weighing unit 109 has changed. For example, the change detection unit 123 detects when the weight has changed by a predetermined value or more within a predetermined time and determines that the weight has changed. Alternatively, the change detection unit 123 may calculate the amount of change in the weight of the product being weighed by the weighing unit 109 and determine that the weight has changed if the amount of change is greater than or equal to a predetermined value. Furthermore, the change detection unit 123 may calculate the amount of change in weight as one aspect of a weight change and detect that the weight is fluctuating. The change detection unit 123 can also detect whether the weighed weight is stable or not. A state of stable weight refers to a state in which, for example, the weight value of the product being measured by the weighing unit 109 falls within a predetermined threshold value within a predetermined minute time. When the change detection unit 123 detects a change in the weight of the weighing unit 109, it is assumed that a product has been placed on the weighing unit 109, and the process from imaging the product by the imaging unit 111 to extracting the product by the extraction unit 125 is initiated.

[0047] <Image Control Unit 124> The imaging control unit 124 controls the imaging unit 111 to image the product placed on the weighing unit 109. The imaging control unit 124 may start imaging for object scanning when the change detection unit 123 detects a change in the weight of the weighing unit 109, or it may start imaging for object scanning when the weight of the product weighed by the weighing control unit 122 is within the transaction weighing range. For example, a range of 20 divisions or more relative to the smallest division is treated as the transaction weight. Alternatively, the operation for object scanning may be started in response to any operation by the operator.

[0048] <Extraction part 125> The extraction unit 125 is a functional unit that extracts product information from the product file 121. For example, it extracts product candidates corresponding to product images captured by the imaging unit 111 from the product file 121, or extracts product information from the product file 121 in response to searches such as product number searches. The extraction unit 125 recognizes products in the image captured by the imaging unit 111 in a so-called object scan using appropriate image analysis or general object recognition technology. Note that general object recognition may be performed by dividing the image into regions for each object. Alternatively, the number or type of products placed on the weighing unit 109 may be identified.

[0049] The extraction unit 125 uses the training images associated with each product in the product file 121 as reference images, compares these reference images with the product images in the captured images recognized by object scanning, and calculates the similarity. The similarity indicates how similar all or part of the product images are to each other, for example, when 100% = "similarity: 1.0" is set to the training images of each product stored in the product file 121. The extraction unit 125 then extracts multiple products from the product file 121 as candidate products corresponding to the captured product images, for example, in order of similarity. The information related to the extracted candidate products is passed to the display control unit 132. Because multiple products are extracted as candidates, the likelihood of selecting the appropriate product increases. As a result, the user is not forced to repeat the extraction process due to the extraction of an inappropriate product candidate. This is particularly useful when the user is a customer, as repeating the process can be confusing and the increased workload is inconvenient. Therefore, extracting multiple product candidates is beneficial.

[0050] The extraction unit 125 may calculate the similarity between the captured product image and the training image of each product registered in the product file 121 as an absolute evaluation, or as a relative evaluation. Furthermore, regarding the extraction of multiple products in order of similarity, the extraction method can be set in various ways. For example, if the similarity of the product ranked first is lower than a predetermined value, three products may be extracted as candidates in order of similarity. Also, if the similarity of the product ranked first is 70%, and there are multiple products within a similarity range of -5% (65%) from that, all products within that range may be extracted as candidates. In addition, it is possible to set the system to extract as candidates any non-saleable products that have a higher similarity than the product ranked first among the products for sale, or if there are non-saleable products within a range of ±3% of the product ranked first in similarity.

[0051] Furthermore, the extraction unit 125 may, for example, start the extraction process in response to weight fluctuations without waiting for the weight of the product to be weighed to stabilize, or it may start the extraction process when a weight fluctuation is detected. Alternatively, the product extraction process may be performed when the weight weighed by the weighing control unit 122 is stable. Furthermore, the extraction unit 125 may identify products with similar color tones by identifying the color of the captured product image during object scanning and comparing it with the color of registered products. The extraction unit 125 may also use color as one of the indicators of similarity and consider products with similar color tones as candidates for captured products. In addition to being identified from the image, the color of the product may also be registered in advance in the product file 121. The similarity of color may be calculated, for example, using each element of hue, lightness, or saturation, or a combination thereof, or any other technique such as projection onto an appropriate coordinate system can be employed.

[0052] Furthermore, the extraction unit 125 may function as follows, particularly in each operating mode. In the first operating mode, the extraction unit 125 extracts candidate products, i.e., candidate products placed in the weighing unit 109 as purchase targets, by referring to the product file 121 based on the image of the product captured by the imaging unit 111. In this case, the extraction unit 125 may first extract candidate products corresponding to the image of the captured product from the products set as products for sale. Since products not for sale should not be displayed, limiting the extraction of candidate products to only products for sale increases the accuracy of extracting appropriate candidates. On the other hand, if products cannot be extracted from the products set as products for sale, candidate products corresponding to the image of the captured product may be extracted from the products set as products not for sale.

[0053] In the second operation mode, the extraction unit 125 extracts candidates for products to which training images should be added, based on the product images captured by the imaging unit 111, by referring to the product file 121. In this case, the extraction unit 125 may extract candidates for products corresponding to the images captured by the imaging unit 111 from all products, regardless of whether they are set as products for sale or not, i.e., from all products. If it is not possible to extract candidates for products to which training images should be added based on the images captured by the imaging unit 111, the extraction unit 125 may extract products for which training images have not been registered in the product file 121, i.e., products for which no training images are associated, as candidates for products to which training images should be newly registered. Furthermore, if it is not possible to extract candidates for products because training images have not been registered in association with products, or because the amount of training is small, i.e., there are few training images associated with the products, a message indicating "Not trained" may be displayed on the screen. Alternatively, the system may determine which state is in place and notify the operator by displaying a message on the screen indicating that there are no training images registered or that there are few training images registered.

[0054] In the third operation mode, product selection can be accepted from the state in which product candidates have been extracted in the first or second operation mode. However, regardless of this, product candidates for sales target setting may be extracted by referring to the product file 121 based on the product image captured by the imaging unit 111, or product candidates may be extracted according to product number search, etc. In this third operation mode, in order to set the sales target setting for products, the extraction unit 125 extracts product candidates from all products regardless of the sales target setting.

[0055] In the fourth operation mode, any number of products are extracted from all products registered in the product file 121 by searching for product numbers or the like. However, when specifying each of the multiple products, as in the other operation modes, candidate products corresponding to the product images captured by the imaging unit 111 may be extracted.

[0056] In addition, in any of the operating modes, during the extraction process by the extraction unit 125, products that do not have a training image associated with them for object scanning, i.e., products for which a training image is not registered in the product file 121, may be excluded from the object scanning, i.e., from the extraction, among all products, including products for sale and products not for sale.

[0057] <Settings section 126> The setting unit 126 sets whether the products registered in the product file 121 are eligible for sale or ineligible for sale. In this embodiment, the setting of whether a product is eligible for sale or ineligible for sale by the setting unit 126 may be referred to as "eligible for sale setting".

[0058] Here, the setting process by the setting unit 126 is performed in the third operation mode, but it can also be performed while the first operation mode is running. That is, while the first operation mode is running, in which products are registered as products to be purchased, the extraction unit 125 extracts product candidates, and when the user requests a change to the third operation mode with the product candidates displayed, the first operation mode is temporarily put on hold and the system switches to the third operation mode. In this third operation mode, the system can set whether the products extracted by the extraction unit 125 are products to be sold or products not to be sold. While operations are being performed in the third operation mode, operations performed in the first operation mode before the operation mode was changed to the third operation mode are kept valid, and when the sales target setting is completed in the third operation mode and the system returns to the first operation mode, the operations that were kept valid can be continued. As will be described later, the system may also switch from the second operation mode or the fourth operation mode to the third operation mode in the same way as switching from the first operation mode to the third operation mode.

[0059] As mentioned above, the operations in the first operation mode that are maintained when switching to the third operation mode include, for example, the weighing value of the product and operations and information related to product candidates. However, if the sales target setting is changed in the third operation mode after transitioning from the first operation mode, the products that should be extracted as candidates based on product similarity may change. For this reason, when returning from the third operation mode to the first operation mode, the extraction process by the extraction unit 125 may be executed again. When the extraction process is executed again in this way, if new product candidates that should be extracted according to similarity arise due to the change in the sales target setting, these new product candidates may be extracted and displayed in addition to the product candidates that were previously extracted in the first operation mode, or a predetermined number of product candidates may be displayed in order of similarity from the previously extracted products and these new product candidates. In either case, in the first operation mode after returning, the results of the new extraction process are reflected in the extraction process results of the previous first operation mode, and the first operation mode can be said to be maintained effectively. However, regardless of this, when transitioning from the first operation mode to the third operation mode and then returning to the first operation mode, the processing results from the previous first operation mode can be erased, and the extraction process by the extraction unit 125, or a series of processes starting from imaging, can be executed from the beginning.

[0060] Furthermore, in the first operation mode, when the extraction unit 125 extracts product candidates from the product file 121 based on the product image captured by the imaging unit 111, if the system is to extract product candidates from products set as products for sale and not from products not for sale, then the sales target setting by the setting unit 126 can be said to be setting whether or not each product is a target for extraction by the extraction unit 125 in the first operation mode (extraction target setting).

[0061] <Registration Section 127> The registration unit 127 registers the product's learning image in the product file 121. Specifically, in the second operation mode, the registration unit 127 registers the product image captured by the imaging unit 111 as a learning image in the product file 121, associating it with the product specified by the designation reception unit 131 (described later). Here, if a learning image has not yet been registered for the product to be associated, it becomes a new registration; if a learning image has already been registered, it becomes an additional registration. Note that the location where the learning image is registered may be a file separate from the product file 121, such as a file (storage area) for registering the learning image, as long as it is associated with a product.

[0062] Furthermore, if the product for which the designation has been accepted is one of the products grouped by the grouping unit 128, the registration unit 127 registers the image of the product captured by the imaging unit 111 as a training image and associates it with all products grouped with the product designated by the designation acceptance unit 131 in the product file 121.

[0063] In the initial stages of registering training images, no training images are registered. In these initial stages, products are retrieved, for example, by searching by product number, and their images are registered as training images. Once a certain amount of training image registration has progressed from the initial stage, product candidates can be extracted by object scanning. By selecting a specific product from the product candidates displayed in response to the captured product image, the training image is associated with the product, and training progresses. In this regard, regardless of the progress of training, if a product candidate completely different from the captured product is extracted as a result of object scanning, the captured product image is deemed unsuitable as a training image. Therefore, a cancel button may be provided to "not register" or "skip" the captured product image as a training image, allowing registration to be canceled. Alternatively, instead of providing a cancel button, or in addition to it, if the product is removed from the weighing unit 109 without the operation of selecting a product being performed, the process of registering the captured product image as a training image may be canceled. This avoids registering training images that would later be deleted as irregular images, unlike cases where training images are registered based on product numbers. In the first place, as the number of registered training images increases and accuracy improves, only products similar to the photographed product will be extracted as candidates, thus preventing images of products with completely different characteristics from the photographed product from being incorrectly associated with the wrong product.

[0064] <Grouping Section 128> The grouping unit 128 groups predetermined products in the fourth operation mode and registers the settings in the product file 121. In this description of the embodiment, the setting by the grouping unit 128 to group products may be referred to as "group setting".

[0065] <Calculation Unit 129> The calculation unit 129 refers to the product file 121 and calculates the price based on the weighed value and unit price of the product. The calculation unit 129 also performs calculations necessary for selling the product, such as calculating the tax-inclusive price based on the consumption tax rate, as appropriate.

[0066] <Print Control Unit 130> The printing control unit 130 issues a label that reflects information such as the weight measured by the weighing unit 109 and the price calculated by the calculation unit 129. In other words, the printing control unit 130 controls the printing unit 110 to print information necessary for product settlement on a label, etc., based on product information about the product in the product file 121, the measured value output as a weighing result from the weighing unit 109, and the price calculated by the calculation unit 129. The printing control unit 130 may, instead of printing the information necessary for product settlement on a label, or in addition to printing it on the bag containing the product. The printing control unit 130 may also present the information to be printed to the operator and issue the label after receiving confirmation that the presented information is correct, or it may issue the label immediately upon generation of the information to be printed without waiting for confirmation. Furthermore, the information necessary for product settlement may be associated with a recording medium that can identify the member or transaction, such as an IC tag or barcode (member card). Before weighing begins or immediately before issuing the label, an input screen for the recording medium's identification information may be displayed to prompt the operator to input the information.

[0067] <Designated Reception Desk 131> The designation reception unit 131 is a means for displaying candidate products extracted from the product file 121 as product candidates corresponding to the product image captured by the imaging unit 111, and for accepting designations from the operator. This designation reception unit 131 is displayed, for example, on the screen operated by the operator and is implemented by an operator control provided for each product. This operator control may be displayed, for example, as a preset key G221 (see Figure 5, etc.) on which product information is displayed, as described later. This allows the operator to easily identify products and select them with simple operations, even if multiple product candidates are extracted.

[0068] In the first operating mode, the designated reception unit 131 accepts the designation of a product for which a label will be issued from the candidate products extracted by the extraction unit 125. In the second operation mode, the designated reception unit 131 accepts the designation of a product for which learning images will be registered, from the candidate products extracted by the extraction unit 125. If no learning images are yet associated with the designated product, it will be treated as a new registration; if learning images are already associated with the product, it will be treated as an additional registration. In the third operation mode, the designated reception unit 131 accepts the designation of a product from the candidate products extracted by the extraction unit 125, and switches the setting of the product to either be a product to be sold or a product to be excluded from sale. In this third mode, the product setting may be switched to either a product to be sold or an excluded product in response to the acceptance of the operation to designate a product. In the fourth operation mode, the designation reception unit 131 accepts the designation of products to be grouped from the product candidates extracted by the extraction unit 125. In the fourth operation mode, the registration of the group may be requested when multiple products are selected from the product candidates and an operation requesting registration is performed. In this case, until the operation requesting registration is performed, it may be possible to select and deselect the products according to the product designation. Furthermore, the operation requesting registration can be performed in various ways, such as pressing a button or a completion key on a mechanical key provided separately on the screen.

[0069] Furthermore, when products are grouped, and the system accepts the selection of a product from the candidate products to be used for registering training images, the selection may be made by product code, or a designated product from the grouped products may be displayed as a representative and the selection may be made from that product. The designated representative product may be, for example, the product with the lowest product code number when the products were grouped, or the product that was first selected when the products were grouped, and is not particularly limited. In addition, when registering training images, only one product from the grouped products may be displayed as a candidate product, and in this case, the one product may be the designated representative product mentioned above.

[0070] <Display Control Unit 132> The display control unit 132 is a functional unit that displays various information, such as product information extracted by the extraction unit 125, on the display touch panel 104. The display control unit 132 displays an operator representing each product extracted as a candidate product corresponding to the image of the product captured by the imaging unit 111 on a designated screen, which will be described later. This operator accepts user selections for each extracted product and is used by the setting unit 126 to set sales targets. For example, it is displayed as a preset key G221 (see Figure 5, etc.) with product information displayed.

[0071] When the display control unit 132 displays operator icons for multiple products extracted by the extraction unit 125 as candidates for captured products on the selection screen, the candidate products shown as operator icons may be displayed, for example, in the order of the product numbers stored in the product file 121. Alternatively, the candidate products may be displayed in the order of similarity calculated by the extraction unit 125. Alternatively, a predetermined number of candidate products with high similarity may be displayed in order of product numbers. Furthermore, candidate products with high similarity may be displayed in a different manner than other candidate products. This allows the user to select a product with high similarity and enables product selection without special operations even if the extraction accuracy by the extraction unit 125 decreases. Note that if multiple candidate products with high similarity have the same similarity, the display manner does not need to differ even if other products with low similarity are displayed. This is because if there are candidate products with the same similarity, the extraction accuracy is considered low, and the user is encouraged to make a more proactive judgment.

[0072] Furthermore, when displaying candidate products indicated as operators in order of similarity, the order does not need to be determined strictly by the degree of similarity. For example, candidate products with a similarity of 90% or higher may be displayed together as candidates with the same degree of similarity without distinguishing their display format. Also, if the extraction unit 125 extracts only products with a similarity of 70% or more but less than 90%, the candidates may not be narrowed down in order of similarity, and the operators for all extracted products may be displayed. This increases the probability that at least one of the operators displayed on the specified screen is appropriate, and eliminates the need to require the operator to specify a product using other special operations.

[0073] Furthermore, if only products with a similarity level below a predetermined threshold, for example less than 70%, are extracted as product candidates, the product controls may not be displayed. Instead, a screen to call a store clerk may be displayed, or a message requesting that a different product of the same type be placed on the scale may be displayed. This is because there is a risk that products that are not on display may be placed on the weighing unit 109, or there may be a malfunction in the object scanning function, such as dust adhering to the lens of the imaging unit 111 or a malfunction in imaging, and therefore inspection is necessary for continued use.

[0074] Furthermore, if the extraction unit 125 fails to extract product candidates, the display control unit 132 may display a product search screen. From the search screen, users can search for products registered in the product file 121 by, for example, product name, product number, department, or classification. The product name and product number may be searchable by either exact match or partial match. The operator can select any product from the search results. Alternatively, if the extraction unit 125 fails to extract product candidates, all products may be displayed on the selection screen and the user can select a product.

[0075] With this configuration, even if the product training image is not stored in the product file 121, or if the extraction unit 125 is unable to extract the product due to reasons such as inability to recognize it using image analysis technology or general object recognition technology, the operator can easily specify the product. The display control unit 132 may also display all product controls separately according to any classification, such as department. The classification may be by color, or by type, such as fruits or vegetables. The display of controls corresponding to each classification may be done through a display area or display page that can be switched by selecting a tab. Furthermore, when a product is object-scanned during the purchase process, the image of the product obtained through the object scan may be registered as a training image in product file 121. Such settings may be optional and their execution may be determined during setup.

[0076] <Screen example (first operation mode)> Here, we will describe an example of a screen displayed by the display control unit 132. Figure 4 shows screen G10 (standby screen) in the first operating mode, where no product is placed on the weighing unit 109 and the unit is waiting to perform weighing and price setting. In the center of screen G10, the words "Place the product in the center of the scale and start" and an illustration illustrating this are displayed, prompting the operator to place the product on the weighing unit 109.

[0077] A product information display area G21 is located above screen G10. The product information display area G21 displays the product's tare weight, weight (or quantity), unit price, and price. The weight field reflects the measurement taken by the weighing unit 109. To display the quantity, the quantity is calculated from the weight and unit weight. The tare weight and unit price are read from the product file 121 when a product is selected by the operator, after the extraction unit 125 has extracted candidate products. The price is calculated by the calculation unit 129 based on the tare weight, weight (or quantity), and unit price when a product is selected by the operator, and then displayed. In the illustrated example, all values ​​are zero because no product is placed on the weighing unit 109.

[0078] Figure 5 shows screen G11 in the first operating mode, where a product is placed on the weighing unit 109 and a selection of product candidates extracted by the extraction unit 125 are displayed. Screen G11 includes the product information display area G21 mentioned above, as well as a preset key display area G22, tabs G23, a text input area G24, a virtual keyboard G25, a numeric input area G26, and a numeric keypad G27.

[0079] In the product information display area G21, the weight measured by the weighing unit 109 is displayed as the weight of the product once it has been placed on the weighing unit 109.

[0080] In the preset key display area G22, a preset key G221, which is an operator for accepting product selection, is displayed as pressable for each product candidate extracted by the extraction unit 125 corresponding to the product image captured by the imaging unit 111. In the first operation mode, the selection of a product for which a label will be issued is accepted in response to the pressing of this preset key G221.

[0081] In the first operation mode, the extraction unit 125 only extracts products that are set as products for sale. Therefore, in the preset key display area G22, only the preset keys G221 corresponding to products set as products for sale are displayed, and those corresponding to products set as products not for sale are not displayed. Furthermore, if any of the products extracted by the extraction unit 125 are grouped, other grouped products may also be extracted, and the preset key G221 of those extracted products may be displayed in the preset key display area G22. However, even in this case, it is preferable not to display products that are set as ineligible for sale.

[0082] Furthermore, in the preset key display area G22, multiple candidate products extracted by the extraction unit 125 are displayed by the preset key G221. However, if products with different similarity levels (ranges of similarity) to the training image are displayed as candidates, the preset key G221 may be displayed in a way that allows for the distinction of similarity levels. For example, if there is one product with a similarity level of 90% or higher and two products with a similarity level of less than 90%, the product with a similarity level of 90% or higher may be displayed in a different manner from the other products to guide the operator's selection. On the other hand, if there are two or more products with a similarity level of 90% or higher, the display manner may not be changed. Also, if there are only candidates with a similarity level of less than 90%, the display manner of the preset key G221 may not be changed, or all preset keys G221 may be displayed without filtering by similarity level.

[0083] Furthermore, the display (arrangement) of the preset key G221 may be in ascending or descending order of product code, or in order of similarity. Also, if there is only one product with a similarity of 90% or more as described above, the display column for the preset key G221 of that product and the other products may be in two rows. That is, if there is only one product with a similarity of 90% or more, the preset key G221 of that product may be displayed in the first row, and the others in the second row, or they may be displayed with a gap equal to the length of one preset key G221. In addition, in cases where only products with a similarity below a predetermined level are extracted as candidates, or when the extraction results of the object scan cannot be obtained, it is conceivable that a product not on display may be placed on the weighing unit 109 due to vandalism or other reasons. For this reason, the operator may be notified, the use may be stopped, or a store employee may be called to prevent the user from placing different individual units of the same product on the weighing unit 109. Notifications can be made in various forms, such as displaying a message on the screen, illuminating a sign light, or outputting sound or a buzzer.

[0084] Furthermore, while the preset key G221 in the illustration displays the product name, other product information registered in product file 121, such as the unit price, may be displayed as appropriate. Additionally, buttons or operations may be associated with the preset key G221 to allow editing of the displayed product information, enabling users to access a screen for editing product information. In this case, it would be preferable to return to the original product selection screen once editing of the product information is complete. Similar buttons and operations may be provided in other operation modes as well.

[0085] Tab G23 consists of Tab G231, which accepts operation mode switching operations, and Tab G232, which allows specifying the classification of each product. Tab G231 accepts switching of operation modes, but the switching operation is not enabled when weighing and pricing products. The switching operation is enabled, for example, when a store employee or other person performs authorization as described below. Once the switching operation is enabled, pressing Tab G231 will switch the operation mode from the first operation mode to another operation mode, such as the third operation mode.

[0086] When a designated tab G232 is selected, the preset keys G221 of products classified under that designated tab G232 are displayed in the preset key display area G22, or the preset keys G221 displayed in the preset key display area G222 are narrowed down to those related to products classified under that designated tab G232. In the diagram, only one tab G232 is provided for specifying product classifications, but multiple tabs may be provided depending on the classification.

[0087] The text input area G24 is an area for entering product information using the virtual keyboard G25, which displays the Japanese syllabary. When a product name is entered in the text input area G24, a search or filtering process is executed that refers to the product file 121. A preset key G221 corresponding to the entered product name is displayed in the preset key display area G22, or the preset keys G221 already displayed in the preset key display area G22 are narrowed down to those corresponding to the product name. Furthermore, the text input area G24 may also allow users to input the product's color as part of the product information. For example, by entering "green" in the text input area G24, the preset key G221 for products associated with the color green will be displayed in the preset key display area G22.

[0088] Numeric input area G26 is an area for entering numerical values ​​related to product information, such as product codes, using the numeric keypad G27. When a product code is entered in the numerical input area G26, the preset key G221 corresponding to the entered product code is displayed in the preset key display area G22, or the preset keys G221 already displayed in the preset key display area G22 are narrowed down to those corresponding to the product code.

[0089] In this screen configuration, the text input area G24 and virtual keyboard G25, and the numerical input area G26 and numeric keypad G27 all constitute a search means for searching for a predetermined product, and a filtering means for narrowing down the search results to products that match predetermined conditions. This makes it possible to access products that are not displayed as preset keys G221 in the preset key display area G22.

[0090] In this first operation mode, when a predetermined preset key G221 is selected from the preset key display area G22, the system transitions to a label issuance confirmation screen for confirming the contents and issuing the label. On the label issuance confirmation screen, for example, the product information display area G21, similar to screen G11, specifically displays the product's tare weight, weight, unit price, and price. If the operator confirms the contents and wishes to issue the label, they press the button to request label issuance. The print control unit 130 controls the print unit 110 accordingly to print the label, and the label is issued. If the operator wishes to cancel label issuance, they may be able to press a cancel button or a button to return to the screen for selecting the product. Label issuance may also be canceled when the product is removed from the weighing unit 109. Notwithstanding these examples, instead of, or in addition to, label issuance, information on the specified product may be transmitted to an external device as registration data. Once label printing is complete and the product is removed from the weighing unit 109, the weighed value becomes zero, and the system returns to screen G10, which is waiting to proceed with printing labels for the next product.

[0091] Furthermore, if the change detection unit 123 detects a change in weight measured by the weighing unit 109 during processing, before the preset key G221 is specified, the product extraction process, and depending on the settings and circumstances, the process may be restarted from imaging, and the product candidates may be displayed again and updated. Also, if the preset key G221 is specified before the product candidates are displayed again in this manner, the product may be treated as having been selected from the product candidates extracted by the process executed before the restart.

[0092] <Screen example (second operation mode)> Figure 6 shows screen G12 in the second operation mode, when registering the image of the captured product as a training image. Screen G12 is an example of a screen that is displayed when a product is placed on the weighing unit 109 and the product is object-scanned, provided that the training image has been registered to a degree that allows for object scanning. Screen G12 displays "Validation Mode" (the system name corresponding to the second operating mode), allowing the operator to understand that the operating mode is the second operating mode. Alternatively, the difference between the operating mode and other operating modes may be made recognizable by the background color, the display of the preset key G221, or other text. However, by keeping the screen configuration largely the same as that of the first operating mode, the operator can perform operations with roughly the same feel as in the first operating mode.

[0093] In the second operation mode, pressing the preset key G221 displayed in the preset key display area G22 allows the selection of a product to which learning images should be added. When a product is selected using the preset key G221, the learning images are associated with the selected product and registered in the product file 121.

[0094] Furthermore, if some of the products extracted by the extraction unit 125 are grouped, only the preset key G221 of the representative product among the grouped products may be displayed in the preset key display area G22. Even if only the preset key G221 of the representative product is displayed in this way, when the specification of the preset key G221 of one product is accepted, the training image is associated with the other grouped products as well, so that even groups of products that are indistinguishable in appearance can be associated with the training image together. Furthermore, the preset keys G221 displayed in the preset key display area G22 in the second operation mode are not limited to those whose corresponding products are set as products for sale, but also include those set as products that are not for sale. In addition, the preset keys G221 displayed in the preset key display area G22 may be displayed in order of product similarity, or in ascending or descending order of product code, as in the first operation mode described above.

[0095] In the second operating mode, pressing tab G231 switches the operating mode from the second to the first operating mode, and tab G231 displays an image that indicates that the operating mode is being switched.

[0096] If a product is object-scanned in the initial stage before any training images for the product are registered, there will be no training images to match the product image in the captured image. In this case, the screen may display, for example, all the preset keys G221 for products registered in product file 121. The operator then specifies the preset key G221 for the product for which they want to register the captured product image as a training image. As a result, the captured product image is associated with the product specified as the training image and registered in product file 121. In this case, the screen displaying the preset key G221 for all products registered in product file 121 may consist of multiple tab-switchable pages. In this case, the user can browse products categorized into a predetermined type by switching tabs and select a specific product from among them. It may also be possible to narrow down the search by product name, or by selecting a department or classification (category). Alternatively, instead of displaying the preset key G221 for all products registered in product file 121 on the screen, product candidates may be displayed as preset key G221 in response to searches such as product name searches.

[0097] Furthermore, when the imaging unit 111 images a product and the extraction unit 125 extracts candidate products, if the product for which a training image is to be registered does not appear among the candidates, all products registered in the product file 121 may be displayed as preset key G221. When a predetermined preset key G221 is specified, the training image is associated with the corresponding product and registered in the product file 121. In this case as well, as described above when training images are registered in the initial stage, the preset key display area G22 may be configured to allow users to narrow down products by switching tabs, and the tabs may be switched not only by department and classification (category) but also depending on whether or not a training image has been registered. This allows users to register learning images for a specific product, regardless of whether or not learning images have already been registered.

[0098] In this second operation mode, if a predetermined preset key G221 is selected from the preset key display area G22, the system will not proceed to the label issuance confirmation screen as in the first operation mode, but will instead provide notification that the learning image has been registered. This notification may be provided by displaying a message on the screen indicating that the learning image has been registered, or by outputting it as audio. It may also be transmitted to another host terminal such as the store controller 2. After a predetermined period of notification, or after an operation such as pressing a button to acknowledge the notification, the system returns to a screen that waits for a product to be placed on the weighing unit 109 in order to register the learning image.

[0099] <Screen example (Third operation mode)> Figure 7 shows screen G13, which is used when setting the target product for sale in the third operation mode. On screen G13, "Status Mode" (the system name corresponding to the third operating mode) is displayed, allowing the operator to understand that the operating mode is the third operating mode. In this example as well, as described above for screen G12, the fact that the operating mode is different from other operating modes may be indicated by the background color or the display of the preset key G221.

[0100] In the third operation mode, pressing the preset key G221 displayed in the preset key display area G22 allows the selection of a product to be accepted, and the sales target setting for that product is switched. In other words, in the third operation mode, the preset key G221 also serves as an operator for setting the sales target by the setting unit 126, and each time the preset key G221 is pressed, the selected product is set to either a sales target product or an ineligible product, and the setting is registered in the product file 121.

[0101] Here, the display of preset key G221 may differ depending on whether the corresponding product is set as a product for sale or a product not for sale. That is, in the illustrated example, preset key G221-1, which represents "kiwi" set as a product for sale, is displayed in blue, and preset key G221-2, which represents "lemon" set as a product not for sale, is displayed in yellow. When preset key G221 is pressed, the change in the product setting is registered accordingly, and the color of preset key G221 also changes to the corresponding color.

[0102] Please note that this color scheme is just one example; any color scheme is acceptable as long as it distinguishes between products for sale and products not for sale. Furthermore, distinction is not limited to color; various methods can be used as long as they distinguish between products for sale and products not for sale. For example, one could be made to flash, or a design or wording that distinguishes them could be attached to each or both. Additionally, products for sale can be grouped together for easier identification, especially when there are many product options.

[0103] Furthermore, the display of preset key G221 in the third operation mode and the display of preset key G221 in the first operation mode may also differ. For example, in the third operation mode, preset key G221 for products that are for sale may be displayed in blue, and those that are not for sale may be displayed in yellow, while preset key G221 displayed in the first operation mode may be displayed in white. This makes it easier to see which operation mode is being used.

[0104] In the preset key display area G22, all preset keys G221 for the extracted products are displayed regardless of whether or not group settings are applied to the products corresponding to preset key G221, and settings can be made individually even for grouped products. Alternatively, if group settings are applied to the extracted products, all preset keys G221 for the grouped products may be displayed in the preset key display area G22.

[0105] Furthermore, the third operation mode may be switchable at the discretion of an operator, such as a store employee, while the first operation mode is running. For example, if, when a product is imaged in the first operation mode and product candidates are displayed by the preset key G221, and the preset key G221 of a certain product that should not be for sale is displayed, the first operation mode is temporarily put on hold and the system switches to the third operation mode. In the preset key display area G22 of the switched third operation mode, the preset key G221 that was displayed in the first operation mode is displayed in a manner that makes it possible to distinguish whether it is set to a product for sale or a product that is not for sale. The operator can change the setting of the certain product from a product for sale to a product that is not for sale and return to the first operation mode. As a result, the preset key G221 of the certain product that was previously displayed as a product for sale is hidden. However, this does not prevent the third operation mode from being executed independently of the first operation mode. For example, the third operation mode can be executed by selecting it from a menu screen independent of each operation mode.

[0106] Furthermore, in the third operating mode, pressing tab G231 switches the operating mode from the third to the first operating mode, and tab G231 displays an image that makes it clear that the operating mode is being switched. In this way, since the target products for sales targeting can be specified from the candidate products extracted by object scanning, the amount of work required is reduced compared to searching for target products by product number from the list of products registered in product file 121 and setting them as sales targets. Furthermore, since sales targeting can be done with an operation feel similar to registering training images in the second operation mode described above, it is easy for the operator to get used to the operation.

[0107] Furthermore, when a new product is displayed in a store and it is a product that has not been displayed before, the store clerk needs to verify whether the product is correctly object-scanned after being displayed. This verification can be done, for example, by checking whether the correct product candidate is displayed as a preset key G221 on screen G11 via object scanning when the product is placed on the weighing unit 109 in the first operation mode. If several other identical items are placed on the weighing unit 109 and appropriate product candidates are extracted as expected, then the preparation is complete. On the other hand, if appropriate product candidates are not extracted, it is necessary to verify the product's sales target setting. At this time, it is convenient for the operator, such as a store clerk, to be able to perform authorization authentication from screen G11, where the weighing value is set, and move to the third operation mode. Authorization may be performed, for example, by scanning the store clerk's barcode with the imaging unit 111, or by entering a password, and is not particularly limited. Also, the transition to the third operation mode is not limited to pressing the tab G231 mentioned above, but may also be triggered by authorization authentication performed during the first operation mode. When switching to the third operating mode, the operator can, as described above, check the sales target setting for each product candidate on screen G13, indicating whether it is a product that can be sold or not, and can also switch that sales target setting.

[0108] <Screen example (Fourth operation mode)> Figure 8 shows screen G14 when grouping products in the fourth operation mode. On screen G14, "Grouping Mode" (the system name corresponding to the fourth operating mode) is displayed, allowing the operator to understand that the operating mode is the fourth operating mode.

[0109] In the fourth operation mode, when multiple products are specified by pressing the preset key G221 displayed in the preset key display area G22, and an operation to request registration is performed, the grouping unit 128 groups the specified products, and the group setting is registered in the product file 121. There is no particular limit to the number of preset keys G221 of the products to be specified; group settings will be accepted as long as there are two or more. Furthermore, until the operation to request registration is performed, products can be selected and deselected according to the specified preset key G221. In addition, the operation to request registration can be performed in various ways, such as pressing a button provided separately on the screen or a completion key on a mechanical key.

[0110] ● Processing Flow Next, the processing flow performed by the weighing device 1 according to this embodiment will be described for each operation mode. When the weighing device 1 is powered on, it displays a menu screen that allows the user to select an operating mode. When a predetermined operating mode is selected from the menu screen, a standby screen for that operating mode is displayed. From the standby screen of each operating mode, it may be possible to return to the menu screen with a predetermined operation. However, a switching mechanism such as an operating mode switching button may be provided on the screen for each operating mode, allowing the user to switch from one operating mode to another depending on the operation of the switching mechanism.

[0111] <Processing Flow 1 (First Operation Mode)> Figure 9 shows the processing flow in the first operating mode. In the standby screen shown in the example of screen G10 above, when an item is placed on the weighing unit 109, the change detection unit 123 detects a change in weight on the weighing unit 109 (S101), and screen G11 is displayed. The weighing control unit 122 continuously performs weighing using the weighing unit 109, and the weighed value is displayed on screen G11. Furthermore, in subsequent processing, when the item is removed from the weighing unit 109 and the weight becomes zero, the system returns to the standby screen G10.

[0112] The imaging unit 111 captures an image of the product placed on the weighing unit 109 in response to a change in weight (S102). Once the weight measured by the weighing unit 109 stabilizes (S103), the extraction unit 125 extracts candidate products corresponding to the imaged product from the product file 121 (S104). Details of the extraction process will be described later.

[0113] The display control unit 132 displays the extracted product candidates on the screen as operators (preset key G221) (S105). When one of the operators is pressed, a product is selected, and the selection reception unit 131 accepts the selection (S106). The calculation unit 129 then calculates the price according to the weight of the product (or the number of units calculated from the unit weight) (S107), and the price is displayed on the screen.

[0114] Once the content of the label to be issued is finalized and confirmation is received from the operator (S108), the print control unit 130 controls the print unit 110 to issue the label (S109).

[0115] Here, Figure 10 shows the detailed flow of the extraction process related to S104 described above. First, the extraction unit 125 attempts to extract candidate products from the products set as products for sale by referring to the product file 121 based on the product image captured by the imaging unit 111 (S111). In the extraction of product candidates, the similarity between the product image and the registered training image is calculated as described above. As a result, if a candidate product corresponding to the captured product can be extracted from the products set as products for sale, that product is extracted as a candidate (S112).

[0116] On the other hand, if no candidate products can be extracted from the products set as products for sale, the extraction unit 125 attempts to extract candidate products from the products set as products not for sale (S113). As a result, if a candidate product corresponding to the imaged product can be extracted from the products set as products not for sale, that product is extracted as a candidate (S112).

[0117] If no candidate products can be extracted from the products set as ineligible for sale (S113), error processing is performed (S114). Error processing may, for example, display a staff call screen to prompt the customer to call a staff member, or display instructions requesting that a different product of the same type be placed on the weighing unit 109. When a staff member is called, the staff member may retrieve the relevant product by searching for the product number, or if the product is placed on the weighing unit 109 again in response to instructions requesting that a different product of the same type be placed on the weighing unit 109, the series of processes from S101 described above will be executed again. Alternatively, instead of such error processing, a message such as "Please select from the preset keys" may be displayed along with a preset key to prompt the customer to specify a product.

[0118] Upon receiving a label, the customer uses the label to complete the payment at the POS terminal 3. The label has a barcode and product code printed on it, which contain payment information such as the product name, price, net weight, unit price, store name, etc., allowing the customer to complete the payment based on the information printed on the label.

[0119] Furthermore, if an item is extracted from the list of items not for sale in S113, this fact may be displayed on the screen, or the display method of the preset key G221 may be changed. Alternatively, the item may not be displayed on the screen as the preset key G221, but rather a mark that is only recognizable to store staff may be displayed to indicate its status. In addition to or instead of a screen display, output such as audio, illumination or flashing of a sign lamp, or notification to other terminals may also be provided. Furthermore, in the illustrated example, if it is not possible to extract product candidates from the products to be sold, an attempt is made to extract product candidates from the products not to be sold. However, in other embodiments, if it is not possible to extract product candidates from the products to be sold, the system may proceed to error processing (S114) without attempting to extract product candidates from the products not to be sold. In this case, if it is possible to extract a product by targeting products set as products not to be sold, the system may notify the staff of this fact by displaying a mark on the screen in a manner that is only understandable to the staff. The manner of notification may be other than those described above.

[0120] Furthermore, if the error handling step S114 is reached, it is possible that, in the extraction of candidate products corresponding to the captured image, there are products associated with training images that have a similarity level above a certain level, but the similarity level is not high enough to display them as candidates, or that no candidates could be extracted from products associated with training images. In such cases, the store staff may be notified of the situation in a manner that allows them to recognize each situation, such as by displaying a message or mark. If it is then determined that no training images have been registered, the relevant product can be immediately retrieved by searching for the product number, without having to go through the maintenance screen or other means to check the status of the product.

[0121] Furthermore, step S104 may be performed simultaneously with step S103, or before the weight stabilizes. However, if step S104 is performed after the conditions of step S103 are met, the similarity can be calculated when the product's posture is stable and stationary, thus ensuring higher identification accuracy.

[0122] <Processing Flow 2 (Second Operation Mode)> Figure 11 shows the processing flow in the second operating mode. Referring to Figure 9, similar to the processing in the first operation mode described above (S101-S103), when an operator such as a store clerk places an item on the weighing unit 109 that they want the weighing device 1 to learn, the change detection unit 123 detects a change in weight on the weighing unit 109 (S201), and the screen G12 described above is displayed. Then, the imaging unit 111 takes an image of the item placed on the weighing unit 109 triggered by the weight change (S202), and waits for the weight being weighed by the weighing unit 109 to stabilize (S203). In this second operation mode as well, the weighing control unit 122 is constantly performing weighing using the weighing unit 109, and the weighed value is displayed on screen G12. When the item is removed from the weighing unit 109 and the weight becomes zero, the system returns to the standby screen.

[0123] Once the weight on the weighing unit 109 stabilizes, the extraction unit 125 attempts to extract candidate products to which training images can be added, based on the captured product image and by referring to the product file 121 (S204). If, for example, a product is identified that has a training image associated with it that matches the captured product image with a similarity of a predetermined degree or higher, the identified product is extracted as a candidate product to which training images can be added (S205).

[0124] On the other hand, if a product cannot be identified that has a training image associated with it that matches the captured product image with a similarity of a predetermined degree or higher, and therefore a candidate product for which a training image can be added cannot be extracted, the captured product is considered to have no training image registered yet, or to have not had enough training images registered to enable a valid object scan, and a product that does not have a training image associated with it is extracted as a candidate product for which a new training image will be registered (S206). In addition, during the processing of S205 and S206, the extraction unit 125 may attempt to extract candidate products from both products set as products for sale and products set as products not for sale.

[0125] The display control unit 132 displays the extracted product candidates on the screen as operators (preset key G221) (S207). When a product is selected by pressing one of the operators, and the selection reception unit 131 accepts the selection (S208), the registration unit 127 acquires the captured image of the product as a training image and registers it in the product file 121 in association with the selected product (S209).

[0126] <Processing Flow 3 (Third Operation Mode)> Figure 12 shows the processing flow in the third operating mode. This section describes the process of switching to the third operation mode while the first operation mode is running and then performing the necessary processing. As described with reference to Figure 9, in step S105, when a candidate product is displayed as preset key G221, and a switch in the operation mode is accepted (S301), the sales target settings for the product are read from product file 121 (S302), and the sales target settings are reflected in the display mode of preset key G221 (S303).

[0127] The operator checks the sales target settings of a product from the display pattern of the preset key G221, and if they determine that a product requires a setting change, they request a change in the sales target settings for that product using the preset key G221 (S304). In response, the setting unit 126 changes the sales target settings of the specified product and registers it in the product file 121 (S305). The setting is reflected in the display pattern of the preset key G221 of the target product in accordance with the change in the sales target settings (S306).

[0128] When the operation to terminate the third operation mode is performed, the operation mode switches back to the first operation mode, and the label is issued after the steps from S106 onwards.

[0129] In the example described above, the system switched from the first operation mode to the third operation mode to set the sales target for each product. However, similar to the first and second operation modes described above, the system may also capture an image of the product from a predetermined standby screen in the third operation mode to display the product's preset key G221, and then switch the sales target setting for each preset key G221. Furthermore, similar to the case where the operation mode is switched from the state in the first operation mode where the preset key G221 is displayed to the third operation mode, it may be possible to switch the operation mode from the state in the second operation mode or the fourth operation mode described later where the preset key G221 is displayed to the third operation mode and set the sales target for each product displayed as the preset key G221.

[0130] <Processing Flow 4 (Fourth Operation Mode)> Figure 13 shows the processing flow in the fourth operating mode. First, on a designated standby screen, products are read from product file 121 and preset key G221 is displayed on the screen (S401). The products displayed as preset key G221 may be extracted by, for example, a product number search, or they may be all products registered in product file 121.

[0131] When the operator selects any preset key G221 from the preset keys G221 displayed on the screen, the selection reception unit 131 accepts it (S402). If multiple preset keys G221 are selected and an operation requesting grouping is performed, such as pressing a predetermined button on the screen (S403), the grouping unit 128 groups the selected items and sets them in the item file 121 (S404).

[0132] Furthermore, the order of the steps in the processing flow of any of the first to fourth operating modes described above can be changed as appropriate, as long as it does not create any inconsistencies.

[0133] According to the weighing device 1 of this embodiment, any product registered in the product file 121 can be registered as a learning image and configured with a general and simple operation. In particular, the process is performed by capturing the product first and then specifying the product number (product), rather than specifying the product number first and then capturing the product image and inputting the image. Therefore, it is possible to register learning images continuously without switching product numbers. Furthermore, simply placing the product on the weighing unit 109 generates a training image using the imaging unit 111. For products for which training images have already been registered, candidates for products to which training images should be registered are automatically extracted by object scanning, thus eliminating the need to specify which products to register training images for.

[0134] Furthermore, even for products for which training images have not yet been registered, the system extracts only products for which training images have not yet been registered as candidates, regardless of whether the operator is aware of whether training images have already been registered or not. This makes it easy to specify a product and to quickly identify if training images have not yet been registered. Furthermore, in the process of registering training images, even if a product other than the intended product is accidentally photographed, only candidate products corresponding to the photographed image are extracted. There is no risk of incorrectly registering the correspondence between training images and products, which significantly reduces the need to delete incorrectly registered training images. Furthermore, the user interface is not a special one; it can be operated while imagining a normal user interface. Furthermore, because designated products are grouped together, it is convenient to register training images for products that have similar appearances or looks all at once.

[0135] Furthermore, it is easy to set sales targeting for products extracted through object scanning. In particular, the extracted products are displayed as selectable preset keys G221, and settings can be easily updated with a touch (press) operation. The settings are displayed in a way that is easy to understand at a glance, which is convenient. In addition, since similar products to the scanned product are also extracted through object scanning, it is useful that the sales targeting settings for similar products can be updated simultaneously. Moreover, settings can be made with the same feel as normal transaction operations, and there is no need to go through the trouble of selecting products as target or excluding them from a product list.

[0136] In this embodiment, if the weighing device 1 is equipped with the functions of a POS terminal 3, or if the information processing device according to the present invention is configured as a POS terminal 3, the POS terminal 3 will perform processes such as extracting products by object scanning, registering products as items to be purchased, and settling payments for registered products. The POS terminal 3 can be any type of POS terminal, including a regular POS terminal where a store clerk performs product registration and settlement, a fully self-service POS terminal where the customer registers and settles products themselves, or a semi-self-service POS terminal where a store clerk registers products using a registration machine and the customer settles payments using a settlement machine. Each POS terminal is equipped with the functional units of a known POS terminal. Specifically, it includes a display / operation unit for displaying and operating various information, a reading unit for reading various codes and acquiring information, a settlement unit for supporting various payment media and cash payments, and a communication unit for enabling data communication with the store controller 2, etc. In this case, the POS terminal 3 may not only register learning images but also, as described above, present multiple product candidates from captured product images as preset keys G221 and accept selections. Furthermore, the settings for target customers and other related parameters may also be handled by POS terminal 3, similar to the method described above.

[0137] Furthermore, in the above embodiment, training images of products sold to customers were registered, but the target of object scanning or the target of registering training images may be items other than such products. For example, containers used in bulk sales (selling by weight) could also be the target. That is, in bulk sales, when selling products such as prepared foods and fruits, multiple sizes of containers are used, and these multiple sizes of containers may be provided to customers by the store at different prices depending on the size. Also, each container has a different tare weight depending on its size. In this case, it would be convenient if the container could be object-scanned when products are placed in the container, and the price of the container could be reflected in the settlement amount, or the tare weight of the container could be subtracted from the weighed value. Therefore, by applying the present invention described above to such a situation, the price, tare weight, and training image could be registered for each size of container, and when registering images of multiple sizes of containers as training images, it would be useful to present the operator with a list of container candidates so that the operator can specify any container.

[0138] Furthermore, depending on the viewpoint, the present invention can be constructed as an invention of a computer program or a method. The computer program can be provided by storing it on various data-readable recording media, or by making it available for download via a network such as the Internet. Furthermore, in other embodiments, as described above, the information processing device according to the present invention can be configured as a POS terminal or a display device. In addition, any device that performs at least image processing, regardless of its name, can constitute the information processing device according to the present invention.

[0139] Furthermore, these software resources can be distributed or consolidated onto any of the hardware resources through appropriate design, and the hardware resources can also be configured as a physically integrated device or as separate devices.

[0140] Specifically, for example, the system can be configured as an information processing system comprising a weighing device 1 and a cloud server 4 positioned as a higher-level device of the weighing device 1. In this case, for example, the weighing device 1 comprises at least a display touch panel 104, a weighing unit 109, an imaging unit 111, a weighing control unit 122, an imaging control unit 124, and a display control unit 132. The cloud server 4 also comprises an extraction unit 125 that extracts product candidates based on the product images captured by the imaging unit 111. Information required by both the weighing device 1 and the cloud server 4 is transmitted and received via a network. That is, the weighing device 1 transmits images of captured products and information specifying products for which learning images will be registered from the product candidates to the cloud server 4, and the cloud server 4 transmits information related to product candidates extracted based on the images of captured products to the weighing device 1.

[0141] <Note> An information processing apparatus according to another embodiment of the present invention comprises: an imaging means for imaging an article; an extraction means for extracting an article corresponding to the article imaged by the imaging means from a predetermined storage means based on similarity with a learning image as a reference image; a designation receiving means for receiving a designation of a predetermined article from one or more articles extracted by the extraction means; and an acquisition means for acquiring images of the predetermined article designated by the designation receiving means and the article imaged by the imaging means as the learning image. If there is no learning image of the article imaged by the imaging means, or if the extraction means cannot extract an article based on the learning image, the designation receiving means accepts a designation from all articles registered in the predetermined storage means, and if an article is extracted by the extraction means based on the image of the article imaged by the imaging means, the designation receiving means accepts a designation from the extracted article.

[0142] In the operation mode for executing a transaction, the system may also include a setting means for setting whether or not to include items extracted by the extraction means based on the learning images of items captured by the imaging means.

[0143] The system may also include a switching means for switching between a first mode and a second mode, wherein the switching means allows the designated receiving means to receive instructions to acquire an image of an item captured by the imaging means as a learning image using the acquisition means in the first mode, and to receive a setting to determine whether to include or exclude the item from the extraction means in the operation to execute the transaction.

[0144] ●Summary of Embodiments This invention relates to an information processing device, an information processing system, and a computer program.

[0145] A technique known as object scanning is known, which identifies an object by comparing the image of the object obtained by photographing the object with a training image that serves as a reference image for image recognition.

[0146] For example, Japanese Patent Publication No. 2017-162217 discloses an article identification device comprising a tray on which articles are placed, a backlight light source that illuminates the bottom surface of the tray with backlight light, a digital camera that images articles on the tray, and an identification computer that performs image recognition on the images from the digital camera and identifies the articles. This article identification device extracts the articles on the tray from a difference image between an image taken when the backlight light source is on and an image taken when the backlight light source is off.

[0147] In image learning for object scanning, it was extremely cumbersome to take an image of each target product, specify the product each time, and register the image.

[0148] Therefore, one of the objectives of the present invention is to provide a device that can easily register training images used for object scanning.

[0149] To achieve the above objective, an information processing device according to one aspect of the present invention comprises: an imaging means for capturing images of products; an extraction means for extracting candidates for products to which learning images are to be added, by referring to a product information storage means that stores product information based on images of products captured by the imaging means; a designation receiving means for receiving designations of products to which the learning images are to be added from the candidate products extracted by the extraction means; and a registration means for registering the images of products captured by the imaging means as learning images in the product information storage means in association with the products designated by the designation receiving means.

[0150] If the extraction means cannot extract candidates for products to which the learning image will be added based on the images of products captured by the imaging means, it may extract products to which the learning image is not associated as candidates for newly registering the learning image, and the designation receiving means may accept a designation of a product to which the learning image will be newly registered from the candidates for products extracted by the extraction means.

[0151] The extraction means may include a display means that, if it is not possible to extract candidate products to which the learning image will be added based on the image of the product captured by the imaging means, displays products stored in the product information storage means in a selectable manner as candidate products to which the learning image will be newly registered, and the designation receiving means may accept a designation of a product to which the learning image will be newly registered from the candidate products displayed by the display means.

[0152] The extraction means may, when the weight of the product measured by the weighing means is within the transaction weighing range, refer to a product information storage means that stores product information based on the image of the product captured by the imaging means to extract candidate products to which the learning images will be added.

[0153] The system further includes a grouping means for grouping predetermined products, and the registration means, if the product for which a designation has been received is one of the products grouped by the grouping means, registers the image captured by the imaging means as the learning image in the product information storage means, associating it with all the products grouped with the product designated by the designation receiving means.

[0154] An information processing system according to one aspect of the present invention is configured such that an information processing device and a higher-level management device can communicate with each other via a network, wherein the information processing device includes an imaging means for capturing images of products, and a designation receiving means for receiving designation of products to which the image of the product captured by the imaging means should be added as a learning image from a list of candidate products, and the management device includes a product information storage means for storing product information, an extraction means for extracting candidate products to which the learning image should be added based on the image of the product captured by the imaging means by referring to the product information storage means, and a registration means for registering the image of the product captured by the imaging means as a learning image in association with the product designated by the designation receiving means in the product information storage means.

[0155] A computer program according to one aspect of the present invention causes a computer to perform an imaging process for capturing images of products; an extraction process for extracting candidate products to which the learning images will be added, by referring to a product information storage means that stores product information based on the images of products captured by the imaging process; a designation acceptance process for accepting designations of products to which the learning images will be added from the candidate products extracted by the extraction process; and a registration process for registering the images of products captured by the imaging process as learning images in the product information storage means in association with the products designated in the designation acceptance process.

[0156] A computer program according to one aspect of the present invention is configured in a system in which an information processing device and a higher-level management device are able to communicate via a network, wherein the information processing device is instructed to perform an imaging process to capture images of products, and a designation acceptance process to accept the designation of a product to which the image of the product captured by the imaging process should be added as a learning image from a list of candidate products, and the management device is instructed to perform an extraction process to extract candidate products to which the learning image should be added, based on the image of the product captured by the imaging process, by referring to a product information storage means that stores product information, and a registration process to register the image of the product captured by the imaging process as a learning image in the product information storage means in association with the product designated in the designation acceptance process.

[0157] According to the article identification device of the present invention as described above, training images used for object scanning can be easily registered. [Explanation of Symbols]

[0158] 1: Measuring device (information processing device) 104: Display touch panel (display means) 109:Measuring part (measuring means) 110: Printing section 111: Imaging unit (imaging means) 121: Product file (product information storage means) 122: Metering Control Unit 123: Change detection unit 124: Imaging control unit 125:Extraction part (extraction means) 126: Setting section (setting means) 127: Registration section (means of registration) 128: Grouping unit (grouping means) 129: Calculation Department 130: Printing Control Unit 131: Designated reception department (designated reception means) 132: Display Control Unit 2: Store Controller 3: POS terminal 4: Cloud Server G21: Product information display area G22: Preset key display area G221: Preset Keys

Claims

1. An imaging means for photographing products, Based on the image of the product captured by the aforementioned imaging means, an extraction means extracts candidate products to which training images will be added by referring to a product information storage means that stores product information, A designation acceptance means that accepts the designation of a product from the candidate products extracted by the extraction means to which the learning image will be added, A registration means that registers the image of the product captured by the imaging means as the learning image, in association with the product specified by the designated reception means, in the product information storage means; If the extraction means fails to extract candidate products to which the learning image will be added based on the image of the product captured by the imaging means, the product search means searches for products stored in the product information storage means as candidates for products to which the learning image will be newly registered. The designated reception means is, The system accepts new registrations of the learning images for products found using the aforementioned product search means. Information processing device.

2. The extraction means is If it is not possible to extract candidate products to which the learning image should be added based on the image of the product captured by the imaging means, then products to which the learning image should not be associated should be extracted as candidates for newly registering the learning image. The designated reception means is, From the candidate products extracted by the extraction means, the system accepts the designation of a product for which the learning image will be newly registered. The information processing apparatus according to claim 1.

3. The extraction means is If, based on the image of the product captured by the imaging means, candidate products to which the learning image can be added cannot be extracted, the system includes a display means that allows selection of products stored in the product information storage means as candidates for newly registering the learning image. The designated reception means is, From the product candidates displayed by the aforementioned display means, the system accepts the selection of a product for which to register the learning image as a new product. The information processing apparatus according to claim 1.

4. The extraction means is When the weight of a product measured by the weighing means falls within the transaction weighing range, the system extracts candidate products to which the learning image will be added by referring to a product information storage means that stores product information based on the image of the product captured by the imaging means. The information processing apparatus according to any one of claims 1 to 3.

5. It further comprises a grouping means for grouping specific products, The registration means is, If the product for which a designation has been accepted is one of the products grouped by the grouping means, the image captured by the imaging means is registered in the product information storage means as the learning image, associating it with all products grouped with the product designated by the designation acceptance means. The information processing apparatus according to any one of claims 1 to 3.

6. The information processing device and the higher-level management device are configured to communicate with each other via a network. The aforementioned information processing device is An imaging means for photographing products, The system includes a designation receiving means that receives a designation of a product from a list of candidate products to which images of the product captured by the imaging means will be added as learning images, The aforementioned control device is A product information storage means for storing product information, Based on the image of the product captured by the imaging means, an extraction means extracts candidate products to which the learning image will be added by referring to the product information storage means, A registration means that registers the image of the product captured by the imaging means as the learning image, in association with the product specified by the designated reception means, in the product information storage means; If the extraction means fails to extract candidate products to which the learning image will be added based on the image of the product captured by the imaging means, the product search means searches for products stored in the product information storage means as candidates for products to which the learning image will be newly registered. The aforementioned designated reception means further, The system accepts new registrations of the learning images for products found using the aforementioned product search means. Information processing system.

7. For computers, The imaging process for capturing images of the product, Based on the image of the product captured by the aforementioned imaging process, an extraction process is performed to extract candidate products to which training images will be added, by referring to a product information storage means that stores product information. From the candidate products extracted by the extraction process, a selection acceptance process accepts the selection of products for which the learning images will be added. A registration process in which the image of the product captured by the aforementioned imaging process is registered in the product information storage means as the learning image, associated with the product specified in the designated reception process, If, in the extraction process, it is not possible to extract candidate products to which the training images will be added based on the images of products captured by the imaging process, then a product search process is executed to search for products stored in the product information storage means as candidates for products to which the training images will be newly registered. The aforementioned designated acceptance process further, The system accepts new registrations of the learning images for products found through the aforementioned product search process. Computer program.

8. In a system in which an information processing device and a higher-level management device are configured to communicate via a network, With respect to the aforementioned information processing device, The imaging process for capturing images of the product, The system executes a selection acceptance process that accepts the selection of a product from among the candidates for products to which the image of the product captured by the aforementioned imaging process will be added as a training image, and With respect to the aforementioned control device, Based on the image of the product captured by the aforementioned imaging process, an extraction process is performed to extract candidate products to which the learning images will be added, by referring to a product information storage means that stores product information. A registration process in which the image of the product captured by the aforementioned imaging process is registered in the product information storage means as the learning image, associated with the product specified in the designated reception process, If, in the extraction process, it is not possible to extract candidate products to which the training images will be added based on the images of products captured by the imaging process, then a product search process is executed to search for products stored in the product information storage means as candidates for products to which the training images will be newly registered. The aforementioned designated acceptance process further, The system accepts new registrations of the learning images for products found through the aforementioned product search process. Computer program.

9. An imaging means for imaging a product, Based on the image of the product captured by the aforementioned imaging means, an extraction means extracts candidate products to which training images will be added by referring to a product information storage means that stores product information, A designation acceptance means that accepts the designation of a product from the candidate products extracted by the extraction means to which the learning image will be added, The system includes a registration means for registering an image of a product captured by the imaging means as a learning image in the product information storage means, associating it with a product specified in the designated reception means. The extraction means is When the weight of a product measured by the weighing means falls within the transaction weighing range, the system extracts candidate products to which the learning image will be added by referring to a product information storage means that stores product information based on the image of the product captured by the imaging means. Information processing device.

Images