Selling price presentation system, information processing apparatus, and program

Abstract

To provide a selling price presentation system, an information processing apparatus, and a program capable of presenting a selling price according to a state of an article which a customer wants to purchase.SOLUTION: A selling price presentation system according to the present invention has: a portable terminal (terminal apparatus) having an imaging unit for capturing an image of an article including appearance thereof, an image output unit for outputting the image captured by the imaging unit to a shop server (information processing apparatus), a selling price acquiring unit for acquiring a selling price of the article in the image from the shop server, and a selling price presenting unit for presenting the selling price of the article by superimposing it on the image; and the shop server having an image acquiring unit for acquiring the image from the portable terminal, an article specifying unit for uniquely specifying the article from the image acquired by the image acquiring unit, a discount amount calculating unit for calculating a discount amount from a normal price of the article based on a shape and a color of appearance of the article, and a selling price outputting unit for outputting the selling price obtained by subtracting the discount amount from the normal price to the portable terminal.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a sales price presentation system, an information processing device, and a program. [Background technology]

[0002] Conventionally, there is known an information processing system that acquires a product code and expiration date from the barcode of a product photographed by a camera equipped on a mobile terminal, and displays a sales price set according to the acquired expiration date on the mobile terminal (for example, Patent Document 1).

[0003] The information processing system in Patent Document 1 is capable of setting sales prices according to best-before dates and expiration dates, but in physical stores, it is desirable to further automate the setting of sales prices according to discoloration and deformation of products such as fresh food products. Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide a sales price suggestion system, an information processing device, and a program that can suggest a sales price according to the condition of an item that a customer wishes to purchase. [Means for solving the problem]

[0005] A sales price presentation system according to an embodiment includes a terminal device and an information processing device. The terminal device includes an imaging unit, an image output unit, a sales price acquisition unit, and a sales price presentation unit. The imaging unit captures an image including the appearance of a product. The image output unit outputs the image captured by the imaging unit to the information processing device. The sales price acquisition unit acquires the sales price of the product shown in the image from the information processing device. The sales price presentation unit presents the sales price of the product by superimposing it on the image. The information processing device also includes an image acquisition unit, a product identification unit, a discount amount calculation unit, and a sales price output unit. The image acquisition unit acquires an image including the appearance of the product from the terminal device. The product identification unit uniquely identifies the product from the image acquired by the image acquisition unit. The discount amount calculation unit calculates a discount amount from the regular price of the product based on the external shape or color of the product. The sales price output unit outputs the sales price obtained by subtracting the discount amount from the regular price to the terminal device. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a sales price presentation system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a schematic configuration of a shopping cart equipped with a mobile terminal provided in the sales price presentation system. [Figure 3] FIG. 3 is a hardware block diagram showing an example of the hardware configuration of a store server included in the sales price presentation system. [Figure 4] FIG. 4 is a diagram illustrating an example of a data structure of the product master stored in the store server. [Figure 5] FIG. 5 is a diagram illustrating an example of the structure of a commodity recognition model stored in the store server. [Figure 6] FIG. 6 is a diagram showing an example of the structure of the shape loss / discoloration determination model stored in the store server. [Figure 7] FIG. 7 is a diagram illustrating an example of the data structure of a sales file stored in the store server. [Figure 8] FIG. 8 is a hardware block diagram showing an example of the hardware configuration of a mobile terminal included in the selling price presentation system. [Figure 9] FIG. 9 is a diagram showing how an image of a product is captured by a mobile terminal. [Figure 10] FIG. 10 is a diagram showing a first example in which the price of a photographed product is displayed on the mobile terminal. [Figure 11] FIG. 11 is a diagram showing a second example in which the price of a photographed product is displayed on the mobile terminal. [Figure 12] FIG. 12 is a diagram showing a third example in which the price of a photographed product is displayed on a mobile terminal. [Figure 13] FIG. 13 is a diagram showing a fourth example in which the price of a photographed product is displayed on the mobile terminal. [Figure 14] FIG. 14 is a functional block diagram illustrating an example of a functional configuration of a sales price presentation system according to the embodiment. [Figure 15] FIG. 15 is a flowchart illustrating an example of the flow of processing performed by the sales price presentation system according to the embodiment. [Figure 16] FIG. 16 is a flowchart showing an example of the flow of the discount amount calculation process. DETAILED DESCRIPTION OF THE INVENTION

[0007] (Outline of the sales price presentation system) A selling price presentation system 10 according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a schematic configuration of the selling price presentation system according to the embodiment.

[0008] As shown in FIG. 1, the selling price presentation system 10 has a configuration in which a store server 20 and a mobile terminal 40 are connected via a communication network N such as a wireless LAN.

[0009] The store server 20 acquires an image captured by the camera 443 (see FIG. 2) provided in the mobile terminal 40 and uniquely identifies a product appearing in the image. The store server 20 also determines a selling price for the identified product. The store server 20 then outputs the determined selling price to the mobile terminal 40. The store server 20 is an example of an information processing device in the present disclosure.

[0010] In response to a customer's operation, the mobile terminal 40 takes an image of the product that the customer wishes to purchase using its own camera 443 (see FIG. 2). The mobile terminal 40 also outputs the captured image of the product to the store server 20. The mobile terminal 40 then acquires the selling price of the product from the store server 20 and displays it on the display device 441 (see FIG. 2) of the mobile terminal 40, superimposed on the image of the product. The mobile terminal 40 is, for example, a smartphone or a tablet terminal. The mobile terminal 40 is an example of a terminal device in the present disclosure.

[0011] When a customer goes shopping at a store such as a retail shop and takes an image of a product he or she wishes to purchase with a camera on his or her own mobile terminal or a mobile terminal loaned by the store, the sales price presentation system 10 displays the sales price of the product by superimposing it on the image of the product taken by the mobile terminal.

[0012] At that time, the sales price presentation system 10 presents the customer with a sales price according to the type of product photographed and the time remaining until the store closes. That is, for products that are preferably sold on the same day, such as prepared foods, the sales price presentation system 10 presents a sales price with a higher discount applied the shorter the time remaining until the store closes.

[0013] Furthermore, the selling price presentation system 10 calculates a selling price according to the discoloration or deformation of the photographed product and presents it to the customer. Specifically, for discolored fresh food products or products with crushed packaging, the selling price presentation system 10 presents a selling price to which a discount amount according to the degree of discoloration or deformation has been applied.

[0014] (General configuration of mobile terminal) The mobile terminal 40 provided in the sales price presentation system will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the schematic configuration of a shopping cart equipped with a mobile terminal provided in the sales price presentation system.

[0015] The shopping cart 15 includes a pair of handles 151, a basket placement portion 152, a plurality of casters 153, and a basket 154 placed on the basket placement portion 152.

[0016] The handle 151 is a part that a customer can grasp to move the shopping cart 15. The basket placement section 152 is a platform on which a basket 154 for storing merchandise is placed. For example, four casters 153 are provided on the legs of the shopping cart 15. Each caster 153 rotates independently, allowing the shopping cart 15 to be pushed by a customer and moved freely within the store.

[0017] A mobile terminal 40 is also installed in the shopping cart 15. The mobile terminal 40 is detachably installed, for example, via an attachment member (not shown) on a connection member that connects the pair of handles 151. The mobile terminal 40 is, for example, a smartphone or a tablet terminal.

[0018] The mobile terminal 40 is operated by the customer himself / herself to perform a product registration process for registering products that the customer wishes to purchase and a cashless payment process for paying for the registered products. Note that the mobile terminal 40 may be configured to perform only the product registration process, and the payment process may be performed by a checkout device (not shown) installed in the store. For example, if the customer wishes to pay in cash, the cash payment is made by the checkout device.

[0019] A customer takes a picture of a product they wish to purchase using the camera 443 installed on the back of the mobile terminal 40. The mobile terminal 40 outputs the captured image to the store server 20. The store server 20 performs image recognition to identify the product shown in the acquired image. The store server 20 then compares the product code (product ID) of the identified product with the product master 222 (see FIG. 4) to read out the price of the product. The store server 20 then outputs the read out price to the mobile terminal 40. The mobile terminal 40 displays the product price acquired from the store server 20 by superimposing it on the image of the product photographed by the customer, which is displayed on the mobile terminal 40. The customer checks the displayed price and, if they wish to purchase the product, operates the mobile terminal 40 to register the product. A more detailed flow of product registration will be described later (see FIGS. 9 to 13).

[0020] When calculating the price of a product, the store server 20 determines the degree of discoloration and deformation of the product from the image of the product acquired from the mobile terminal 40. The store server 20 also determines whether the current time falls within the time period for the limited-time sale and whether the target product is eligible for the limited-time sale. The store server 20 then determines a discount amount according to whether the target product is eligible for the limited-time sale, whether the product has discolored or lost its shape, and calculates the selling price with the discount amount applied.

[0021] After completing the registration of all products that the customer desires to purchase, the customer operates the mobile terminal 40 to select a payment method and carry out payment processing.

[0022] (Store server hardware configuration) The hardware configuration of the store server 20 will be described with reference to Fig. 3. Fig. 3 is a hardware block diagram showing an example of the hardware configuration of the store server provided in the sales price presentation system.

[0023] The store server 20 has a configuration in which a control unit 21, a storage unit 22, and a communication controller 24 are connected to one another via an internal bus 23.

[0024] The control unit 21 controls the overall operation of the store server 20. The control unit 21 includes a CPU (Central Processing Unit) 211, a ROM (Read Only Memory) 212, and a RAM (Random Access Memory) 213. The CPU 211 is connected to the ROM 212 and the RAM 213 via internal buses such as an address bus and a data bus. The CPU 211 loads various programs stored in the ROM 212 and the storage unit 22 into the RAM 213. The CPU 211 controls the operation of the store server 20 by operating in accordance with the various programs loaded into the RAM 213. In other words, the control unit 21 has the configuration of a general computer. Note that the configuration of the control unit 21 is not limited to this, and for example, the CPU 211 may be directly connected to the ROM 212 and the RAM 213 without the internal bus 23.

[0025] The storage unit 22 is a storage device such as an HDD or SSD. Alternatively, the storage unit 22 may be a non-volatile memory such as a flash memory that retains stored information even when the power is turned off. The storage unit 22 stores a control program 221, a product master 222, a product recognition model 223, a shape deformation / discoloration determination model 224, and a sales file 225.

[0026] The control program 221 is a program that controls the overall operation of the store server 20. The control program 221 may be provided in a state stored in the storage unit 22, or may be provided by being recorded in an installable or executable file on a computer-readable non-transitory recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD. The control program 221 may also be stored on a computer connected to a network and provided by being downloaded via the network. Furthermore, the control program 221 may be provided or distributed via a network such as the Internet.

[0027] The product master 222 is a master file that stores product information such as product names and prices of products sold in the store. The product master 222 stores product names, unit prices, additional information, and the like in association with product codes. The contents of the product master are updated as needed, so the store server 20 constantly updates the contents of the product master 222 to the latest state by, for example, acquiring the latest product master from a higher-level server (not shown) via the communication controller 24. The data structure of the product master 222 will be described in detail later (see FIG. 4).

[0028] The product recognition model 223 is a model used to uniquely identify a product from an image of the product's appearance acquired from the mobile terminal 40. The structure of the product recognition model 223 will be described in detail later (see FIG. 5). In addition to the product recognition model 223, a code information recognition model may be provided that, when code information (e.g., a barcode) appears in the image of the product's appearance, recognizes information (e.g., a product code) registered in the code information.

[0029] The shape deformation / discoloration determination model 224 is a model used to determine the degree of shape deformation and discoloration of a product shown in an image of the product's appearance obtained from the mobile terminal 40. The structure of the shape deformation / discoloration determination model 224 will be described in detail later (see FIG. 6).

[0030] The sales file 225 is a file that stores payment details for products for which a customer has made payment using the mobile terminal 40. The mobile terminal 40 outputs the payment details to the store server 20 each time the payment process is completed. The store server 20 then stores the payment details acquired from the mobile terminal 40 in the sales file 225. The data structure of the sales file 225 will be described in detail later (see FIG. 7).

[0031] The control unit 21 also performs various communications with the mobile terminal 40 via the communication controller 24. Specifically, the store server 20 acquires an image of the exterior of a product from the mobile terminal 40. The store server 20 also acquires from the mobile terminal 40 a product registration file 422 (see FIG. 8) indicating the product registration result and a payment file 423 (see FIG. 8) indicating the payment result. The store server 20 also outputs the price of the product to the mobile terminal 40.

[0032] (Product master data structure) The data structure of the product master 222 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the data structure of the product master stored in the store server.

[0033] The product master 222 is a master file that stores product information such as product names and prices of products for sale. The product master 222 stores information such as "product classification," "product name," "whether the product is on sale," "regular price," "expiration date / best before date," and "discount range," in association with a product ID (e.g., product code) that uniquely identifies the product.

[0034] "Product category" is information indicating the category name to which the product belongs, for example, food products, miscellaneous goods, etc.

[0035] "Product name" is the name of the product associated with the product code indicated by the product ID.

[0036] "Is this product eligible for limited-time sale?" is information indicating whether the product is eligible for limited-time sale.

[0037] "Regular price" is the regular price of the product associated with the product code indicated by the product ID.

[0038] "Use by date / Best before date" is information indicating the expiration date or best before date of a product, for example, for a product classified as a food product. Note that the "Use by date / Best before date" may be registered in the code information printed or affixed to the product.

[0039] "Discount range" is information indicating the discount range of the product associated with the product code indicated by the product ID. The discount range is information indicating, for example, the discount amount in the case of a limited-time sale, the discount amount according to the discoloration of the exterior color of the product such as fresh food, and the discount amount according to the deformation of the exterior shape of the product.

[0040] For example, if the product is a prepared food item and is subject to a limited-time sale, information such as "limited-time sale discount amount 5-30% off" is stored in the "available discount range" field. Note that the closer it is to the store's closing time, the larger the limited-time sale discount amount may be set. In this case, the current time and the limited-time sale discount amount can be stored in association with each other in the "available discount range" field.

[0041] In addition, in the discount range field, when the product is a perishable food product, information such as "5 to 30% discount depending on discoloration" is stored as the discount amount depending on the discoloration of the appearance.

[0042] Furthermore, in the discount range column, for products that may lose their shape, information such as "5 to 30% discount depending on the loss of shape" is stored.

[0043] Depending on the product, multiple discount items may be applied, including "time sale," "discoloration of exterior color," and "deformation of shape."

[0044] (Product Recognition Model Structure) The structure of the merchandise recognition model 223 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the structure of a merchandise recognition model stored in the store server.

[0045] The commodity recognition model 223 is, for example, a model described by a known neural network. The commodity recognition model 223 is a model that has previously learned a huge number of appearance images of various types of commodities through supervised learning (machine learning). The example shown in Fig. 5 is an example in which the commodity recognition model 223 is configured as a neural network with a three-layer structure having an input layer La, a hidden layer Lb, and an output layer Lc.

[0046] The input layer La has multiple nodes, and each node receives multiple different feature quantities extracted from product appearance images. The input layer La multiplies the multiple different feature quantities by a predetermined coefficient and outputs the resulting integrated value to the intermediate layer Lb.

[0047] The intermediate layer Lb, also known as a convolutional layer, has multiple nodes, and each node performs a convolution operation, which is the sum of the integrated values ​​calculated in the input layer La. The intermediate layer Lb then outputs the results of the convolution operation to the output layer Lc.

[0048] The output layer Lc calculates the sum of the results of the convolution operations input from the intermediate layer Lb. The results calculated by the output layer Lc indicate the products that are determined to correspond to the product appearance images input to the product recognition model 223. Note that the output value of the output layer Lc may be a vector quantity whose elements are probabilities representing the likelihood of each product (for example, a Q-dimensional vector indicating the likelihood of product A, product B, ..., product Q). In this case, the number of dimensions is the number of products to be identified.

[0049] The values ​​of the coefficients to be multiplied by the output of the input layer La and the values ​​of the coefficients to be multiplied by the output of the intermediate layer Lb are determined by repeatedly performing supervised learning.

[0050] Furthermore, the neural network that constitutes the commodity recognition model 223 is not limited to a three-layer structure. Furthermore, the commodity recognition model 223 may be a model in a form other than a neural network, such as a template that stores images of the appearances of multiple commodities photographed from different angles.

[0051] (Structure of the shape and discoloration judgment model) The structure of the shape deformation / discoloration determination model 224 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the structure of the shape deformation / discoloration determination model stored in the store server.

[0052] The shape deformation / discoloration determination model 224 is, for example, a model described by a known neural network. The shape deformation / discoloration determination model 224 is a model that has previously learned the degree of shape deformation and the degree of discoloration in appearance of a huge number of appearance images of various products by performing supervised learning (machine learning). The example shown in FIG. 6 is an example in which the shape deformation / discoloration determination model 224 is configured as a three-layer neural network having an input layer Ld, an intermediate layer Le, and an output layer Lf.

[0053] The input layer Ld has multiple nodes, and each node receives multiple different feature quantities extracted from an appearance image of a product and multiple different feature quantities extracted from an ideal appearance image of the product. The input layer Ld outputs the integrated values ​​obtained by multiplying the multiple different feature quantities by predetermined coefficients to the intermediate layer Le.

[0054] The intermediate layer Le, also known as a convolutional layer, has multiple nodes, each of which performs a convolution operation, i.e., calculates the sum of the integrated values ​​calculated in the input layer Ld. The intermediate layer Le then outputs the results of the convolution operation to the output layer Lf.

[0055] The output layer Lf calculates the sum of the results of the convolution operations input from the intermediate layer Le. The sum calculated by the output layer Lf indicates the degree of deformation and discoloration of the product input to the shape deformation / discoloration determination model 224.

[0056] The neural network that constitutes the shape deformation / discoloration determination model 224 is not limited to a three-layer structure. The shape deformation / discoloration determination model 224 may also be a model in a form other than a neural network, such as a template that stores images of the appearances of multiple products photographed from different angles.

[0057] In addition, in this embodiment, the degree of deformation of the product and the degree of discoloration of the product's exterior color are determined using one model, but the degree of deformation of the product and the degree of discoloration of the exterior color may also be determined using separate models.

[0058] (Sales file data structure) The data structure of the sales file 225 will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the data structure of the sales file stored in the store server.

[0059] The sales file 225 is generated based on the product registration file 422 (see Figure 8) and the settlement file 423 (see Figure 8) output from the mobile terminal 40 in association with the transaction ID, and the product appearance image output from the mobile terminal 40.

[0060] The sales file 225 stores sales information indicating the details of sales in association with a transaction ID. The sales information includes, for example, the date and time of payment, the product ID, the product name, the regular price, the sale price, and an image of the product's appearance.

[0061] The payment date and time is the date and time when the payment was made by the mobile terminal 40. The payment method may also be stored together with the payment date and time.

[0062] The product ID is the product code of the purchased product.

[0063] The product name is the name of the product associated with the product code indicated by the product ID.

[0064] The regular price is the regular price of the product associated with the product code indicated by the product ID.

[0065] The sales price is the selling price of the relevant product. The sales price is the regular price of the product minus the discount amount.

[0066] The external image of the product is an external image of the product taken by the camera 443 of the mobile terminal 40. The external image of the product is stored as evidence that the selling price of the product is reasonable.

[0067] The information stored in the sales file 225 is not limited to the information shown in Fig. 7. For example, if a customer is a registered member, the sales file 225 may store a customer ID.

[0068] (Mobile device hardware configuration) The hardware configuration of the mobile terminal 40 will be described with reference to Fig. 8. Fig. 8 is a hardware block diagram showing an example of the hardware configuration of the mobile terminal included in the selling price presentation system.

[0069] The mobile terminal 40 has a configuration in which a control unit 41, a storage unit 42, a peripheral device controller 44, and a communication controller 45 are connected to one another via an internal bus 43.

[0070] The control unit 41 controls the overall operation of the mobile terminal 40. The control unit 41 includes a CPU 411, a ROM 412, and a RAM 413. The CPU 411 is connected to the ROM 412 and the RAM 413 via internal buses such as an address bus and a data bus. The CPU 411 loads various programs stored in the ROM 412 and the storage unit 42 into the RAM 413. The CPU 411 controls the operation of the mobile terminal 40 by operating in accordance with the various programs loaded into the RAM 413. In other words, the control unit 41 has the configuration of a general computer. Note that the configuration of the control unit 41 is not limited to this, and for example, the CPU 411 may be directly connected to the ROM 412 and the RAM 413 without the internal bus 43.

[0071] The storage unit 42 is a storage device such as an HDD or SSD. Alternatively, the storage unit 42 may be a non-volatile memory such as a flash memory that retains stored information even when the power is turned off. The storage unit 42 stores a control program 421, a product registration file 422, and a payment file 423.

[0072] The control program 421 is a program that controls the overall operation of the mobile terminal 40. The control program 421 may be provided in a state stored in the storage unit 42, or may be provided by being recorded in an installable or executable file on a computer-readable non-transitory recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD. The control program 421 may also be provided by being stored on a computer connected to a network and downloaded via the network. Furthermore, the control program 421 may be provided or distributed via a network such as the Internet.

[0073] The product registration file 422 is a file that stores registered product information (for example, product name, quantity, unit price, price, etc.) in association with a transaction code (hereinafter referred to as a transaction ID) that identifies one transaction.

[0074] The payment file 423 is a file that stores payment details (for example, payment date and time, payment method, payment details, payment amount, etc.) related to the transaction in association with the transaction ID.

[0075] Furthermore, the control unit 41 is connected via a peripheral device controller 44 to a display device 441, an operation device 442, and a camera 443, which input and output various types of information.

[0076] The display device 441 is a display configured with, for example, a liquid crystal panel or an organic EL panel, and displays various image information, text information, etc. related to the transaction.

[0077] The operation device 442 is, for example, a touch panel stacked on the display device 441. The operation device 442 acquires operation information of the operator on the mobile terminal 40 and outputs the information to the control unit 41.

[0078] Camera 443 has an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and captures an image in the direction in which it is pointed. Camera 443 is installed on the rear side of display device 441 of mobile terminal 40.

[0079] The control unit 41 also performs various communications with the store server 20 via the communication controller 45. Specifically, the mobile terminal 40 outputs an image of the exterior of a product captured by the camera 443 to the store server 20. The mobile terminal 40 also acquires the price of the product shown in the image from the store server 20. The mobile terminal 40 also outputs a product registration file 422 and a payment file 423 to the store server 20.

[0080] (Example of information displayed on a mobile device) Examples of information displayed on display device 441 of mobile terminal 40 will be described using Figures 9 and 10. Figure 9 is a diagram showing how an image of a product is taken with the mobile terminal. Figure 10 is a diagram showing a first example in which the price of the photographed product is displayed on the mobile terminal.

[0081] When a customer finds a desired product Ma in the store, the customer removes the mobile terminal 40 from the shopping cart 15, points the camera 443 at the desired product Ma, and taps the activation button (not shown) of the camera 443. When the activation button of the camera 443 is tapped, the mobile terminal 40 displays the image captured by the camera 443 on the display device 441.

[0082] Next, the customer taps the capture button 69 (not shown) displayed on the display device 441. When the capture button 69 is tapped, the mobile terminal 40 captures an image of the product Ma displayed on the display device 441 at that time.

[0083] The mobile terminal 40 outputs the captured image of the product Ma to the store server 20. When the store server 20 acquires the image of the product Ma from the mobile terminal 40, it performs a product recognition process to identify the product Ma. After identifying the product Ma, the store server 20 reads the price of the product from the product master 222. Then, the store server 20 outputs the price of the product Ma to the mobile terminal 40.

[0084] When the mobile terminal 40 acquires the price of the product Ma from the store server 20, the mobile terminal 40 displays the price of the product Ma on the display device 441, superimposing the price on the image of the product Ma.

[0085] FIG. 10 shows an example of a state in which the price of product Ma is displayed on the mobile terminal 40.

[0086] At this time, the display device 441 displays a price display 61, a registration button 65, a checkout button 66, and a back button 68 superimposed on the image of the product Ma.

[0087] The price display 61 is a display showing the price of the product Ma obtained from the store server 20.

[0088] The registration button 65 is an operator that is tapped when registering the product Ma displayed on the screen. When the registration button 65 is tapped, the mobile terminal 40 registers the product Ma in the product registration file 422 (see FIG. 8).

[0089] The checkout button 66 is an operator that is tapped when making a payment for a registered product. When the checkout button 66 is tapped, the mobile terminal 40 performs a payment process related to the registered contents of the product registration file 422.

[0090] The back button 68 is an operator to be tapped when the user does not wish to purchase the product Ma. When the back button 68 is tapped, the mobile terminal 40 transitions to a state in which the image captured by the camera 443 is displayed on the display device 441 (the state in FIG. 9).

[0091] (Example of product price displayed on a mobile device) Examples of product prices displayed on display device 441 of mobile terminal 40 will be described using Fig. 11 to Fig. 13. Fig. 11 is a diagram showing a second example in which the price of a photographed product is displayed on the mobile terminal. Fig. 12 is a diagram showing a third example in which the price of a photographed product is displayed on the mobile terminal. Fig. 13 is a diagram showing a fourth example in which the price of a photographed product is displayed on the mobile terminal.

[0092] FIG. 11 shows an example of a state in which the price of product Ma reflecting a discount due to a limited-time sale is displayed on the mobile terminal 40.

[0093] At this time, the display device 441 displays a price display 61, a limited time sale display 62, a registration button 65, a checkout button 66, and a back button 68 superimposed on the image of the product Ma.

[0094] The price display 61, the registration button 65, the checkout button 66, and the back button 68 are as described above (see FIG. 10).

[0095] The limited-time sale display 62 indicates that the price displayed in the price display 61 reflects a discount due to a limited-time sale.

[0096] FIG. 12 shows an example of a state in which the price of the product Ma is displayed on the mobile terminal 40, reflecting a discount according to the degree of discoloration of the product.

[0097] At this time, the display device 441 displays a price display 61, a color change display 63, a registration button 65, a checkout button 66, and a back button 68 superimposed on the image of the product Ma.

[0098] The price display 61, the registration button 65, the checkout button 66, and the back button 68 are as described above (see FIG. 10).

[0099] The discoloration indicator 63 indicates that the price displayed on the price indicator 61 reflects a discount due to the discoloration of the product.

[0100] FIG. 13 shows an example of a state in which the price of the product Mb is displayed on the mobile terminal 40, reflecting a discount corresponding to the product's deformation.

[0101] At this time, the display device 441 displays a price display 61, a shape distortion display 64, a registration button 65, a checkout button 66, and a back button 68 superimposed on the image of the product Mb.

[0102] The price display 61, the registration button 65, the checkout button 66, and the back button 68 are as described above (see FIG. 10).

[0103] The shape loss indication 64 indicates that the price displayed in the price indication 61 reflects a discount due to the product's shape loss.

[0104] Depending on the condition of the product, multiple discounts may be applied: a limited-time sale discount, a discount due to discoloration, and a discount due to shape loss. In such cases, the display device 441 of the mobile terminal 40 displays multiple corresponding discounts from among the limited-time sale display 62, the discoloration display 63, and the shape loss display 64.

[0105] (Functional configuration of the sales price presentation system) The functional configuration of the sales price presentation system 10 will be described with reference to Fig. 14. Fig. 14 is a functional block diagram showing an example of the functional configuration of the sales price presentation system according to the embodiment.

[0106] The control unit 21 of the store server 20 deploys the control program 221 in the RAM 213 and runs it to realize, as functional units, an image acquisition unit 31, a product identification unit 32, a discount amount calculation unit 33, a sales price output unit 34, and a sales data acquisition unit 35, all of which are shown in Fig. 14. Note that some or all of these functional units may be realized by dedicated hardware.

[0107] The image acquisition unit 31 acquires an image of a product from the mobile terminal 40.

[0108] The product identification unit 32 uniquely identifies the product ID (for example, product code) of the product shown in the image acquired by the image acquisition unit 31.

[0109] The discount amount calculation unit 33 calculates the discount amount from the regular price of the product based on the product's external shape or color. More specifically, the discount amount calculation unit 33 calculates the discount amount from the regular price of the product on sale based on the time when the product image was acquired from the terminal device and the store's closing time. The discount amount calculation unit 33 also calculates the discount amount from the regular price of the product based on the degree of difference in the external shape or color between the image acquired by the image acquisition unit 31 and an image of the product's appearance in a normal state stored by the discount amount calculation unit 33. Note that the discount amount calculation unit 33 preferably sets a larger discount amount for the product on sale as the time approaches the store's closing time. The discount amount calculation unit 33 also preferably sets a larger discount amount as the product becomes more discolored or loses its shape.

[0110] The sales price output unit 34 outputs to the mobile terminal 40 the sales price obtained by subtracting the discount amount from the regular price.

[0111] The sales data acquisition unit 35 acquires the product registration file 422 and the settlement file 423 from the mobile terminal 40.

[0112] Furthermore, the control unit 41 of the mobile terminal 40 deploys the control program 421 in the RAM 413 and operates it to realize, as functional units, the imaging unit 51, the image output unit 52, the selling price acquisition unit 53, the selling price presentation unit 54, the product registration unit 55, and the payment processing unit 56 shown in Fig. 14. Note that some or all of these functional units may be realized by dedicated hardware.

[0113] The imaging unit 51 captures an image including the appearance of the product.

[0114] The image output unit 52 outputs the image captured by the imaging unit 51 to the store server 20.

[0115] The sales price acquisition unit 53 acquires from the store server 20 the sales prices of the products shown in the images captured by the imaging unit 51.

[0116] The selling price presentation unit 54 presents the selling price of the product by superimposing it on the image of the product.

[0117] The product registration unit 55 performs a product registration process to register the selected product in the product registration file 422.

[0118] The payment processing unit 56 performs a payment process to pay the price for the product registered in the product registration file 422. Furthermore, when the payment process is completed, the payment processing unit 56 generates a payment file 423 indicating the payment result.

[0119] (Processing flow of the sales price presentation system) The flow of processing performed by the sales price presentation system 10 will be described with reference to Figures 15 and 16. Figure 15 is a flowchart showing an example of the flow of processing performed by the sales price presentation system of the embodiment. Figure 16 is a flowchart showing an example of the flow of discount amount calculation processing.

[0120] First, the flow of the process performed by the mobile terminal 40 will be described.

[0121] A customer shopping at a store operates the mobile terminal 40 to select "start product registration" from a main menu (not shown), which puts the mobile terminal 40 in a state where product registration can be performed (step S11).

[0122] The imaging unit 51 activates the camera 443 and captures an image of the product (step S12).

[0123] The imaging unit 51 displays the image captured by the camera 443 on the display device 441 (step S13).

[0124] The imaging unit 51 detects that a capture button 69 (not shown) displayed on the display device 441 has been tapped, and captures an image of the commodity Ma displayed on the display device 441 at that time. Then, the image output unit 52 outputs the image captured by the imaging unit 51 to the store server 20 (step S14).

[0125] The selling price acquisition unit 53 acquires, from the store server 20, the selling price of the product Ma that appears in the image captured by the imaging unit 51 (step S15).

[0126] The selling price presenting unit 54 presents the selling price of the product Ma acquired from the store server 20, superimposed on the image of the product Ma (step S16).

[0127] The product registration unit 55 determines whether the product Ma has been registered (step S17). If it is determined that the product Ma has been registered (step S17: Yes), the process proceeds to step S18. On the other hand, if it is not determined that the product Ma has been registered (step S17: No), the process returns to step S12. Note that the product registration unit 55 determines that the product Ma has been registered on the condition that the registration button 65 displayed on the display device 441 has been tapped.

[0128] If it is determined in step S17 that the product Ma has been registered, the product registration unit 55 updates the product registration file 422 by additionally registering information related to the product Ma in the product registration file 422 (step S18).

[0129] Next, the payment processing unit 56 determines whether the checkout button 66 displayed on the display device 441 has been tapped (step S19). If it is determined that the checkout button 66 has been tapped (step S19: Yes), the process proceeds to step S20. On the other hand, if it is not determined that the checkout button 66 has been tapped (step S19: No), the process returns to step S12.

[0130] If it is determined in step S19 that the checkout button 66 has been tapped, the payment processing unit 56 performs payment processing (step S20) according to the details registered in the product registration file 422. The payment method is selected by the customer operating the mobile terminal 40.

[0131] The payment processing unit 56 outputs a payment file 423 indicating the payment result to the store server 20 (step S21). At this time, the product registration file 422 is also output to the store server 20. Thereafter, the mobile terminal 40 ends the process.

[0132] Next, the flow of processing performed by the store server 20 will be described.

[0133] The image acquisition unit 31 acquires an image of the product Ma from the mobile terminal 40 (step S31).

[0134] The product identification unit 32 uniquely identifies the product code of the product Ma shown in the image acquired by the image acquisition unit 31 (step S32).

[0135] The discount amount calculation unit 33 performs a discount amount calculation process to calculate a discount amount from the regular price of the product Ma shown in the image (step S33). The detailed flow of the discount amount calculation process performed in step S33 will be described later (see FIG. 16).

[0136] The sales price output unit 34 outputs the sales price obtained by subtracting the discount amount from the regular price to the mobile terminal 40 (step S34).

[0137] The sales data acquisition unit 35 acquires information indicating the details of the payment (for example, the product registration file 422 and the payment file 423) from the mobile terminal 40 (step S35).

[0138] The sales data acquisition unit 35 updates the sales file 225 based on the information indicating the payment details acquired in step S35 (step S36). After that, the store server 20 ends the process.

[0139] Next, the flow of the discount amount calculation process performed by the store server 20 in step S33 of FIG. 15 will be described with reference to FIG.

[0140] The discount amount calculation unit 33 determines whether the time when the store server 20 acquired an image containing an image of product Ma from the mobile terminal 40 was during a time sale period at the store (step S41). If it is determined that the time was during a time sale period (step S41: Yes), the process proceeds to step S42. On the other hand, if it is not determined that the time was during a time sale period (step S41: No), the process proceeds to step S44. The discount amount calculation unit 33 may also determine whether the time when the image, which the mobile terminal 40 outputs together with the image of product Ma, was captured was during a time sale period at the store.

[0141] If it is determined in step S41 that it is a time period during the limited-time sale, the discount amount calculation unit 33 determines whether the identified product Ma is a product eligible for the limited-time sale (step S42). If it is determined that the product Ma is a product eligible for the limited-time sale (step S42: Yes), the process proceeds to step S43. On the other hand, if it is not determined that the product Ma is a product eligible for the limited-time sale (step S42: No), the process proceeds to step S44. The discount amount calculation unit 33 determines whether the product Ma is a product eligible for the limited-time sale by referring to the product master 222.

[0142] If it is determined in step S42 that the product Ma is a product eligible for the limited-time sale, the discount amount calculation unit 33 adds a discount according to the limited-time sale that is registered in association with the product Ma in the product master 222 (step S43). The discount amount for the limited-time sale may be set higher as the time approaches the closing time of the store.

[0143] Next, the discount amount calculation unit 33 analyzes the image of the product Ma and determines whether the product Ma is out of shape (step S44). If it is determined that the product Ma is out of shape (step S44: Yes), the process proceeds to step S45. On the other hand, if it is not determined that the product Ma is out of shape (step S44: No), the process proceeds to step S46. The discount amount calculation unit 33 determines whether the product Ma is out of shape using the shape deformation / discoloration determination model 224.

[0144] If it is determined in step S44 that the product Ma is out of shape, the discount calculation unit 33 adds a discount according to the degree of out of shape that is registered in association with the product Ma in the product master 222 (step S45). At this time, if a discount due to a limited-time sale has already been added in step S43, the discount due to the limited-time sale and the discount due to out of shape are added together.

[0145] Next, the discount amount calculation unit 33 analyzes the image of the product Ma and determines whether the product Ma is discolored (step S46). If it is determined that the product Ma is discolored (step S46: Yes), the process proceeds to step S47. On the other hand, if it is not determined that the product Ma is discolored (step S46: No), the process proceeds to step S48. The discount amount calculation unit 33 determines whether the product Ma is discolored using the shape deformation / discoloration determination model 224.

[0146] If it is determined in step S46 that the product Ma has discoloration, the discount calculation unit 33 adds a discount according to the degree of discoloration that is registered in association with the product Ma in the product master 222 (step S47). At this time, if a discount due to a limited-time sale and a discount due to deformation have already been added, a discount due to discoloration is further added.

[0147] The discount amount calculation unit 33 determines the discount amount for the product Ma by adding the discount according to the limited-time sale, the discount according to the degree of deformation, and the discount according to discoloration (step S48). Then, the process returns to step S34 in Fig. 15. Note that if the discount according to the limited-time sale, the discount according to the degree of deformation, and the discount according to discoloration are all applied, the discount amount may become large, so it is desirable to set a maximum discount amount (for example, a maximum discount amount of 60% off the product price) in advance.

[0148] (Effects of the embodiment) As described above, the sales price presentation system 10 of this embodiment includes a mobile terminal 40 (terminal device) including an imaging unit 51 that captures an image including the appearance of a product, an image output unit 52 that outputs the image captured by the imaging unit 51 to the store server 20 (information processing device), a sales price acquisition unit 53 that acquires the sales price of the product shown in the image from the store server 20, and a sales price presentation unit 54 that superimposes the sales price on the image and presents the product's sales price, and the store server 20 includes an image acquisition unit 31 that acquires an image from the mobile terminal 40, a product identification unit 32 that uniquely identifies the product from the image acquired by the image acquisition unit 31, a discount amount calculation unit 33 that calculates a discount amount from the regular price of the product based on the product's external shape or color, and a sales price output unit 34 that outputs the sales price obtained by subtracting the discount amount from the regular price to the mobile terminal 40. Therefore, a sales price according to the condition of the product that the customer wishes to purchase can be presented. Furthermore, since the store clerk does not need to attach discount labels indicating the discount amount to each item, the workload of the store clerk can be reduced.

[0149] Furthermore, in the sales price presentation system 10 of this embodiment, the discount amount calculation unit 33 further calculates the discount amount from the regular price of the product based on the time when the image was acquired from the mobile terminal 40 (terminal device) and the closing time of the store. Therefore, there is no need to attach labels indicating the discount amount of the product during the time sale to each product, which reduces the workload of the store clerk.

[0150] Furthermore, in the sales price presentation system 10 of this embodiment, the sales price output unit 34 further outputs information on the basis of the discount amount calculated by the discount amount calculation unit 33 to the mobile terminal 40 (terminal device), and the sales price presentation unit 54 further presents the information on the basis of the discount amount by superimposing it on the image. Therefore, the customer can easily understand the reason why the discount amount was set.

[0151] Furthermore, in the selling price presentation system 10 of this embodiment, the discount amount calculation unit 33 calculates the discount amount according to the discoloration or shape loss of the product using the shape loss / discoloration determination model 224 that has been trained to output the degree of discoloration and shape loss of the product in response to the input of the image acquired by the image acquisition unit 31 and an image of the product's appearance in a normal state. Therefore, the discount amount according to the discoloration or shape loss of the product can be easily calculated.

[0152] The store server 20 (information processing device) of this embodiment also includes an image acquisition unit 31 that acquires an image including the appearance of a product from a mobile terminal 40 (terminal device), a product identification unit 32 that uniquely identifies the product from the image acquired by the image acquisition unit 31, a discount amount calculation unit 33 that calculates a discount amount from the regular price of the product based on the external shape or color of the product, and a sales price output unit 34 that outputs the price obtained by subtracting the discount amount from the regular price to the mobile terminal 40. Therefore, a sales price can be set according to the condition of the product that a customer wishes to purchase.

[0153] Although the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as set forth in the claims. [Explanation of symbols]

[0154] 10 Sales price display system 15 Shopping Cart 20 Store server (information processing device) 222 Product Master 223 Product Recognition Model 224 Shape and discoloration judgment model 225 Sales File 31 Image acquisition unit 32 Product Specification Department 33 Discount calculation section 34 Selling price output section 35 Sales Data Acquisition Department 40 Mobile terminal (terminal device) 443 Camera 51 Imaging unit 52 Image output unit 53 Sales Price Acquisition Department 54 Sales Price Presentation Department 55 Product Registration Department 56 Payment processing section 61 Price Display 62 Time Sale Display 63 Discoloration display 64 Shape distortion display 65 Registration button 66 Checkout button 68 Back button 69 Shooting button Ma,Mb Product [Prior art documents] [Patent documents]

[0155] [Patent Document 1] Japanese Patent Publication No. 2022-008075

Claims

1. an imaging unit that captures an image including the appearance of the product; an image output unit that outputs the image captured by the imaging unit to an information processing device; a sales price acquisition unit that acquires the sales price of the product shown in the image from the information processing device; a terminal device including a sales price display unit that displays the sales price of the product by superimposing it on the image; an image acquisition unit that acquires the image from the terminal device; a product identification unit that uniquely identifies the product from the image acquired by the image acquisition unit; a discount amount calculation unit that calculates a discount amount from a regular price of the product based on the external shape or external color of the product; and an information processing device including a sales price output unit that outputs a sales price obtained by subtracting the discount amount from the regular price to the terminal device. Sales price display system.

2. The discount amount calculation unit further calculates a discount amount from the regular price of the product based on the time when the image was acquired from the terminal device and the closing time of the store. The sales price suggestion system according to claim 1 .

3. The sales price output unit further outputs information on the basis of the discount amount calculated by the discount amount calculation unit to the terminal device, The selling price presentation unit further presents information on the basis of the discount amount by superimposing it on the image.

3. The selling price presentation system according to claim 1 or 2.

4. the discount amount calculation unit calculates a discount amount according to the deformation or discoloration of the product using a model trained to output the degree of discoloration and the degree of deformation of the product in response to input of the image acquired by the image acquisition unit and an image of the product's appearance in a normal state; The sales price suggestion system according to claim 1 .

5. an image acquisition unit that acquires an image including an external appearance of a product from a terminal device; a product identification unit that uniquely identifies the product from the image acquired by the image acquisition unit; a discount amount calculation unit that calculates a discount amount from a regular price of the product based on the external shape or external color of the product; a sales price output unit that outputs a price obtained by subtracting the discount amount from the regular price to the terminal device, Information processing device.

6. A computer that controls a sales price presentation system that includes a terminal device and an information processing device, The terminal device, an imaging unit that captures an image including the appearance of the product; an image output unit that outputs the image captured by the imaging unit to an information processing device; a sales price acquisition unit that acquires the sales price of the product shown in the image from the information processing device; a selling price display unit that displays the selling price of the product by superimposing it on the image, The information processing device an image acquisition unit that acquires the image from the terminal device; a product identification unit that uniquely identifies the product from the image acquired by the image acquisition unit; a discount amount calculation unit that calculates a discount amount from a regular price of the product based on the external shape or external color of the product; a sales price output unit that outputs a sales price obtained by subtracting the discount amount from the regular price to the terminal device, program.

Images