Sales price presentation system, information processing apparatus, and storage medium

Abstract

The invention discloses a sales price presentation system, an information processing apparatus, and a storage medium capable of presenting a sales price according to a state of a commodity that a customer wants to purchase. The sales price presentation system comprises a mobile terminal serving as a terminal device and a shop server serving as an information processing device. The mobile terminal includes an image capturing unit that captures an image including an appearance of a commodity, and an image output unit that outputs the image captured by the image capturing unit to the information processing device. A sales price acquisition unit that acquires, from the information processing device, the sales price of the commodity captured in the image; and a sales price presentation unit that presents the sales price of the commodity by superimposing the sales price on the image. The store server includes an image acquisition unit that acquires an image from the terminal device, and a commodity designation unit that uniquely designates a commodity on the basis of the image acquired by the image acquisition unit. A discount amount calculation unit that calculates a discount amount based on the normal price of the product on the basis of the appearance shape or the appearance color of the product; and a sales price output unit that outputs a sales price obtained by subtracting the discount amount from the normal price to the terminal device.

Description

[0001] This application claims priority from Japanese Application No. JP 2024-066597, filed April 17, 2024, and the contents of the above application are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to a sales price presentation system, an information processing apparatus, and a storage medium. BACKGROUND

[0003] At present, an information processing system is known which acquires a product code and a shelf life from a bar code of a product photographed by a camera equipped in a mobile terminal, and displays a sales price set in accordance with the acquired shelf life on the mobile terminal.

[0004] Although the setting of a sales price in accordance with a shelf life or a use-by date is possible in the existing information processing system, it is also desired to automatically set a sales price in accordance with discoloration of a product such as a fresh food product or deformation of a product in a physical store. SUMMARY

[0005] In view of the above problem, the present application aims to provide a sales price presentation system, an information processing apparatus, and a storage medium which are capable of presenting a sales price in accordance with a state of a product a customer wishes to purchase.

[0006] To solve the above problem, the sales price presentation system of an embodiment includes a terminal apparatus and an information processing apparatus. The terminal apparatus includes an imaging section, an image output section, a sales price acquisition section, and a sales price presentation section. The imaging section photographs an image including an appearance of a product. The image output section outputs the image photographed by the imaging section to the information processing apparatus. The sales price acquisition section acquires a sales price of the product photographed in the image from the information processing apparatus. The sales price presentation section presents the sales price of the product superimposed on the image. Further, the information processing apparatus includes an image acquisition section, a product specification section, a discount amount calculation section, and a sales price output section. The image acquisition section acquires the image including the appearance of the product from the terminal apparatus. The product specification section uniquely specifies the product in accordance with the image acquired by the image acquisition section. The discount amount calculation section calculates a discount amount based on a normal price of the product based on an appearance shape or an appearance color of the product. The sales price output section outputs a sales price obtained by subtracting the discount amount from the normal price to the terminal apparatus.

[0007] According to the above-described sales price presentation system, it is possible to present a sales price in accordance with a state of a product a customer wishes to purchase.

[0008] In the sales price prompting system described above, the discount amount calculation section further calculates the discount amount based on the normal price of the product based on a time at which the image is acquired from the terminal device and a closing time of the store.

[0009] According to the sales price prompting system described above, the workload of the store clerk can be reduced because it is not necessary to attach a label or the like showing the discount amount of the product in the time-limited price reduction on each product.

[0010] In the sales price prompting system described above, the sales price output section further outputs information that is a basis for the discount amount calculated by the discount amount calculation section to the terminal device, and the sales price prompting section further prompts the information that is a basis for the discount amount superimposed on the image.

[0011] According to the sales price prompting system described above, the customer can easily understand the reason for setting the discount amount.

[0012] In the sales price prompting system described above, the discount amount calculation section uses a model trained to output a degree of discoloration and a degree of deformation of the product in response to an input of an image acquired by the image acquisition section and an image of the appearance of the product in a normal state, and calculates the discount amount in accordance with the deformation or discoloration of the product.

[0013] According to the sales price prompting system described above, the discount amount can be easily calculated in accordance with the discoloration or deformation of the product.

[0014] In the sales price prompting system described above, the discount amount calculation section calculates the discount amount for the product within a predetermined discountable range.

[0015] According to the sales price prompting system described above, the discount can be performed within a limited range.

[0016] In the sales price prompting system described above, the terminal device further includes a product registration section that performs product registration processing of the product prompted by the sales price prompting section in accordance with an operation of a registration button.

[0017] According to the sales price prompting system described above, the sales price obtained by subtracting the discount amount of the product can be confirmed and the product registration processing can be performed.

[0018] In the sales price presentation system described above, the discount amount calculation section calculates a discount amount based on the degree of deformation or the degree of discoloration of the product using a model trained to output the degree of discoloration and the degree of deformation of the product in response to an input of an image obtained by the image obtaining section and the appearance of the product in a normal state, the sales price output section outputs the degree of discoloration and the discount amount or the degree of deformation and the discount amount of the product to the terminal device, and the sales price presentation section further presents the degree of discoloration and the discount amount corresponding thereto or the degree of deformation and the discount amount corresponding thereto superimposed on the image.

[0019] According to the sales price presentation system described above, the basis of the discount amount can be presented explicitly.

[0020] In the sales price presentation system described above, the sales price output section further outputs information indicating that the product cannot be sold with respect to a product whose degree of discoloration is equal to or greater than a prescribed degree of discoloration or a product whose degree of deformation is equal to or greater than a prescribed degree of deformation, and the sales price presentation section further presents a situation in which the product cannot be sold when the information indicating that the product cannot be sold is obtained.

[0021] According to the sales price presentation system described above, it is possible to prevent a product whose quality has significantly deteriorated from being registered.

[0022] The information processing apparatus according to another aspect of the present application includes an image obtaining section that obtains an image containing the appearance of a product from a terminal device, a product specifying section that uniquely specifies the product based on the image obtained by the image obtaining section, a discount amount calculation section that calculates a discount amount based on the normal price of the product based on the shape or the color of the appearance of the product, and a sales price output section that outputs a price obtained by subtracting the discount amount from the normal price to the terminal device.

[0023] According to the information processing apparatus described above, it is possible to set a sales price in accordance with the state of a product that a customer wishes to purchase.

[0024] The storage medium of another aspect of the present application stores a program that causes a computer that controls a sales price presentation system including a terminal device and an information processing device to control, so that the terminal device and the information processing device function as: the terminal device functions as: an imaging section that captures an image including an appearance of a product; an image output section that outputs the image captured by the imaging section to the information processing device; a sales price acquisition section that acquires a sales price of the product captured in the image from the information processing device; and a sales price presentation section that presents the sales price of the product superimposed on the image, and the information processing device functions as: an image acquisition section that acquires the image from the terminal device; a product specification section that uniquely specifies the product based on the image acquired by the image acquisition section; a discount amount calculation section that calculates a discount amount based on a normal price of the product based on an appearance shape or an appearance color of the product; and a sales price output section that outputs, to the terminal device, a sales price obtained by subtracting the discount amount from the normal price.

[0025] According to the above-described storage medium, it is possible to provide a function to a computer, and it is possible to present a sales price according to a state of a product that a customer wants to purchase. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a diagram showing an example of a schematic structure of a sales price presentation system of an embodiment.

[0027] Figure 2 is a diagram showing an example of a schematic structure of a shopping cart of a mobile terminal included in a sales price presentation system.

[0028] Figure 3 is a hardware block diagram showing an example of a hardware structure of a store server included in a sales price presentation system.

[0029] Figure 4 is a diagram showing an example of a data structure of a product master file stored in a store server.

[0030] Figure 5 is a diagram showing an example of a structure of a product identification model stored in a store server.

[0031] Figure 6 is a diagram showing an example of a structure of a metamorphosis discoloration determination model stored in a store server.

[0032] Figure 7 is a diagram showing an example of a data structure of a sales file stored in a store server.

[0033] Figure 8is a hardware block diagram showing an example of a hardware structure of a mobile terminal included in the sales price presentation system.

[0034] Figure 9 is a diagram showing a state in which an image of a product is captured by a mobile terminal.

[0035] Figure 10 is a diagram showing a first example in which a price of a captured product is displayed on a mobile terminal.

[0036] Figure 11 is a diagram showing a second example in which a price of a captured product is displayed on a mobile terminal.

[0037] Figure 12 is a diagram showing a third example in which a price of a captured product is displayed on a mobile terminal.

[0038] Figure 13 is a diagram showing a fourth example in which a price of a captured product is displayed on a mobile terminal.

[0039] Figure 14 is a functional block diagram showing an example of a functional structure of the sales price presentation system of the embodiment.

[0040] Figure 15 is a flowchart showing an example of a flow of processing performed by the sales price presentation system of the embodiment.

[0041] Figure 16 is a flowchart showing an example of a flow of discount amount calculation processing.

[0042] LEGEND

[0043] 10 sales price presentation system 15 shopping cart 20 store server (information processing device)

[0044] 222 product master file 223 product identification model 224 deformation discoloration determination model

[0045] 225 sales file 31 image acquisition section 32 product specification section 33 discount amount calculation section

[0046] 34 sales price output section 35 sales data acquisition section 40 mobile terminal (terminal device)

[0047] 443 camera 51 imaging section 52 image output section 53 sales price acquisition section

[0048] 54 sales price presentation section 55 product registration section 56 settlement processing section

[0049] 61 price display 62 time-limited price reduction display 63 discoloration display 64 deformation display

[0050] 65 registration button 66 checkout button 68 return button 69 photographing button

[0051] Ma, Mb goods DETAILED DESCRIPTION

[0052] (Outline structure of sales price prompting system)

[0053] USAGES Figure 1 The sales price prompting system 10 as an embodiment of the present application will be described. Figure 1 is a diagram showing an example of an outline structure of the sales price prompting system of the embodiment.

[0054] As shown in Figure 1 , the sales price prompting system 10 has a structure in which the store server 20 and the mobile terminal 40 are connected by a communication network N such as a wireless LAN.

[0055] The store server 20 acquires an image photographed by the camera 443 (see Figure 2 ) equipped in the mobile terminal 40, and uniquely specifies a photographed good in the image. Further, the store server 20 determines a sales price of the specified good. Then, the store server 20 outputs the determined sales price to the mobile terminal 40. The store server 20 is an example of the information processing device in the present disclosure.

[0056] The mobile terminal 40 photographs an image of a good that a customer wants to purchase with the camera 443 (see Figure 2 ) equipped in the mobile terminal 40 according to an operation of the customer. Further, the mobile terminal 40 outputs the photographed image of the good to the store server 20. Then, the mobile terminal 40 acquires a sales price of the good from the store server 20, and causes it to be superimposed on and displayed on the image of the good on the display device 441 (see Figure 2 ) of the mobile terminal 40. The mobile terminal 40 is, for example, a smartphone, a tablet terminal, or the like. The mobile terminal 40 is an example of the terminal device in the present disclosure.

[0057] The sales price prompting system 10, when a customer shops at a store such as a retail store, causes a sales price of a good that the customer wants to purchase to be superimposed and displayed on an image of the good photographed with a camera equipped in a mobile terminal brought by the customer or borrowed from the store when the customer photographs the good with the camera.

[0058] At this time, the sales price prompting system 10 prompts a sales price to the customer according to a type of the photographed good and a remaining time until the store closes. That is, for a good that is expected to be sold within the day, such as a prepared food, the less the remaining time until the store closes, the higher the discount amount applied to the sales price prompted by the sales price prompting system 10.

[0059] Further, the sales price presentation system 10 calculates the sales price and presents it to the customer based on the discoloration or deformation of the photographed product. Specifically, for a fresh food product that has been discolored, a product whose package has been damaged, and the like, the sales price presentation system 10 presents the sales price to which a discount amount is applied based on the degree of discoloration and the degree of deformation.

[0060] (Summary structure of mobile terminal)

[0061] The use of Figure 2 The mobile terminal 40 included in the sales price presentation system will be described. Figure 2 is a view showing a summary structure of a shopping cart equipped with the mobile terminal included in the sales price presentation system.

[0062] The shopping cart 15 includes a pair of handles 151, a shopping basket placement portion 152, a plurality of casters 153, and a shopping basket 154 placed on the shopping basket placement portion 152.

[0063] The handles 151 are portions that are held by the customer and move the shopping cart 15. The shopping basket placement portion 152 is a table on which the shopping basket 154 that stores products is placed. For example, four casters 153 are provided at leg portions of the shopping cart 15. By individually rotating each caster 153, the shopping cart 15 can be freely moved within the store by the customer pushing it.

[0064] Further, the mobile terminal 40 is provided on the shopping cart 15. For example, the mobile terminal 40 is detachably provided on a connection member that connects the pair of handles 151 via a mounting member (not shown). The mobile terminal 40 is, for example, a smartphone or a tablet terminal.

[0065] The mobile terminal 40 is operated by the customer himself / herself to perform a product registration process of registering products that the customer wishes to purchase and a non-cash settlement process of paying for the amount of money related to the registered products. Alternatively, the mobile terminal 40 can be a structure that performs only the product registration process, and the settlement process is performed by an accounting device (not shown) provided in the store. For example, if the customer wishes to settle in cash, the cash settlement is performed by the accounting device.

[0066] The customer photographs the product that the customer wishes to purchase using the camera 443 provided on the back of the mobile terminal 40. The mobile terminal 40 outputs the photographed image to the store server 20. The store server 20 specifies the photographed product in the acquired image by performing image recognition. Then, the store server 20 associates the product code (product ID) of the specified product with the product master file 222 (see FIG. 2) and outputs the product code to the mobile terminal 40. Figure 4) to read out the price of the product. Thereafter, the store server 20 outputs the read price to the mobile terminal 40. The mobile terminal 40 displays the price of the product obtained from the store server 20 by superimposing it on the image of the product photographed by the customer and displayed on the mobile terminal 40. The customer confirms the displayed price and, if he wishes to purchase the product, registers the product by operating the mobile terminal 40. In addition, a more detailed process for product registration will be described later (see Figures 9 to 13 ).

[0067] Furthermore, when calculating the price of a product, the store server 20 determines the degree of discoloration and deformation of the product based on the product image obtained from the mobile terminal 40. Furthermore, the store server 20 determines whether the current time falls within the time period of the limited-time offer and whether the product is eligible for the limited-time offer. The store server 20 then determines a discount amount based on whether the product is eligible for the limited-time offer, whether the product is discolored, or whether it is deformed, and calculates the resulting sales price after applying the discount.

[0068] After completing the registration of all commodities that the customer wishes to purchase, the customer selects a payment method and performs the payment process by operating the mobile terminal 40 .

[0069] (Store server hardware structure)

[0070] use Figure 3 The hardware structure of the store server 20 will be described. Figure 3 This is a hardware block diagram showing an example of the hardware configuration of a store server included in the sales price presentation system.

[0071] The store server 20 has a configuration in which a control unit 21 , a storage unit 22 , and a communication controller 24 are connected to one another via an internal bus 23 .

[0072] The control section 21 controls the operation of the store server 20 as a whole. The control section 21 includes a CPU (Central Processing Unit) 211, a ROM (Read Only Memory) 212, and a RAM (Random Access Memory) 213. The CPU 211 is connected to the ROM 212 and the RAM 213 via an internal bus of an address bus, a data bus, and the like. The CPU 211 loads various programs stored in the ROM 212, the storage section 22, into the RAM 213. The CPU 211 controls the operation of the store server 20 by operating in accordance with the various programs loaded in the RAM 213. That is, the control section 21 has the structure of a general-purpose computer. In addition, the structure of the control section 21 is not limited to this, and, for example, the CPU 211 and the ROM 212, the RAM 213 are not directly connected via the internal bus 213.

[0073] The storage section 22 is a storage device such as an HDD, an SSD, or the like. In addition, the storage section 22 can be a nonvolatile memory such as a flash memory that retains stored information even when the power is turned off. The storage section 22 stores a control program 221, a product master file 222, a product recognition model 223, a deformation discoloration determination model 224, and a sales file 225.

[0074] The control program 221 is a program that controls the operation of the store server 20 as a whole. The control program 221 can be provided in a state of being stored in the storage section 22, or can be stored in a computer-readable non-transitory storage medium such as a CD-ROM, a floppy disk (FD), a CD-R, a DVD, or the like in an installable form or an executable form and provided. In addition, the control program 221 can be provided by being stored in a computer connected to a network and downloaded via the network. Also, the control program 221 can be provided or distributed via a network such as the Internet.

[0075] The product master file 222 is a master file that stores product information such as product names, prices, and the like of products sold in the store. In the product master file 222, product names, unit prices, and additional information, and the like are stored in association with product codes. In addition, since the contents of the product master file are updated at any time, the store server 20 always updates the contents of the product master file 222 to the latest state by, for example, acquiring the latest product master file from a higher-level server (not illustrated) via the communication controller 24. In addition, the data structure of the product master file 222 will be described later in detail (see Figure 4 ).

[0076] The product recognition model 223 is a model used when uniquely specifying a product in accordance with an image of the appearance of the product acquired from the mobile terminal 40. The structure of the product recognition model 223 will be described later in detail (seeFigure 5 In addition, a code information recognition model that recognizes information (e.g., a product code) registered in code information (e.g., a bar code) in the case where the code information is also captured in the image of the product appearance can be included in addition to the product recognition model 223.

[0077] The deformation discoloration determination model 224 is a model used when determining the degree of deformation or discoloration of a product captured in the image of the product appearance acquired from the mobile terminal 40. The structure of the deformation discoloration determination model 224 will be described later in detail (see Figure 6 ).

[0078] The sales file 225 is a file in which the settlement contents of the product for which the settlement process has been performed by the customer using the mobile terminal 40 are stored. The mobile terminal 40 outputs the settlement contents to the store server 20 each time the settlement process is completed. Then, the store server 20 saves the settlement contents acquired from the mobile terminal 40 in the sales file 225. The data structure of the sales file 225 will be described later in detail (see Figure 7 ).

[0079] Further, the control section 21 also performs various communications with the mobile terminal 40 via the communication controller 24. Specifically, the store server 20 acquires the image in which the product appearance is captured from the mobile terminal 40. Further, the store server 20 also acquires the product registration file 422 indicating the registration result of the product (see Figure 8 ) and the settlement file 423 indicating the result of the settlement (see Figure 8 ) from the mobile terminal 40. Further, the store server 20 outputs the price of the product to the mobile terminal 40.

[0080] (Data structure of product master file)

[0081] The data structure of the product master file 222 will be described using Figure 4 . Figure 4 is a diagram indicating an example of the data structure of the product master file stored in the store server.

[0082] The product master file 222 is a master file in which product information such as the product name or the price of the product to be sold is stored. In the product master file 222, "product classification", "product name", "is the product an object of time-limited price reduction?", "normal price", "effective period / shelf life", "discountable range", and the like are stored in association with the product ID (e.g., product code) uniquely specifying the product.

[0083] The "product classification" is information indicating the classification name to which the product belongs, for example, food, sundries, and the like.

[0084] "Commodity name" is the name of the commodity associated with the commodity code indicated by the commodity ID.

[0085] "Is it a target of time-limited markdown?" is information indicating whether the commodity is a target of time-limited markdown.

[0086] "Normal price" is the normal price of the commodity associated with the commodity code indicated by the commodity ID.

[0087] "Effective period / shelf life" is information indicating the effective period or shelf life of the commodity for a commodity of which the commodity category is, for example, food. In addition, the "effective period / shelf life" can be registered in the code information printed or adhered on the commodity.

[0088] "Discountable range" is information indicating the discountable range of the commodity associated with the commodity code indicated by the commodity ID. The discountable range is, for example, the discount amount in the case of time-limited markdown, the discount amount corresponding to the discoloration of the appearance color of the commodity such as fresh food, and the predetermined discount amount corresponding to the deformation of the appearance of the commodity.

[0089] For example, if the commodity is cooked food and is a target of time-limited markdown, information such as "the discount amount of time-limited markdown is 5% to 30% reduction" is stored in the column of discountable range. In addition, the discount amount of time-limited markdown can be set to be larger as the time closer to the closing time of the store. In this case, the current time and the discount amount of time-limited markdown can be stored in the column of discountable range in association with each other.

[0090] Further, if the commodity is fresh food or the like, information such as "the discount amount corresponding to discoloration is 5% to 30% reduction" is stored in the column of discountable range as the discount amount corresponding to the discoloration of the appearance color. The degree of discoloration and the discount amount are stored in association with each other. For example, the degree of discoloration is 5% to 10% and the discount amount is stored as 5% reduction, the degree of discoloration is 11% to 15% and the discount amount is stored as 20% reduction, the degree of discoloration is 16% to 20% and the discount amount is stored as 30% reduction. In addition, if the degree of discoloration is 21% or more, information of prohibition of sale is stored. The discount is made within the range of discountability according to the degree of discoloration.

[0091] Also, for a commodity in which deformation on the appearance of the commodity can occur, information such as "the discount amount according to deformation is 5% to 30% reduction" is stored in the column of discountable range. The degree of deformation and the discount amount are stored in association with each other. For example, the degree of deformation is 5% to 10% and the discount amount is stored as 5% reduction, the degree of deformation is 11% to 15% and the discount amount is stored as 20% reduction, the degree of deformation is 16% to 20% and the discount amount is stored as 30% reduction. In addition, if the degree of deformation is 21% or more, information of prohibition of sale is stored. The discount is made within the range of discountability according to the degree of deformation.

[0092] Depending on the product, multiple discount items including "Limited-time price reduction", "Appearance color change", and "Transformation" may be applied.

[0093] (Structure of product recognition model)

[0094] use Figure 5 The structure of the commodity recognition model 223 will be described. Figure 5 This is a diagram showing an example of the structure of a product recognition model stored in a store server.

[0095] The product recognition model 223 is a model described in, for example, a well-known neural network. The product recognition model 223 is a model that has been previously trained on a large number of appearance images of various types of products through supervised learning (machine learning). Figure 5 The illustrated example is an example in which the commodity recognition model 223 is composed of a neural network having a three-layer structure of an input layer La, an intermediate layer Lb, and an output layer Lc.

[0096] The input layer La has multiple nodes, and multiple different feature values ​​extracted from the product appearance image are input to each node. The input layer La outputs the cumulative value obtained by cumulatively adding predetermined coefficients to the multiple different feature values ​​to the intermediate layer Lb.

[0097] The intermediate layer Lb, also called a convolutional layer, has multiple nodes, each of which performs a so-called convolution operation to calculate the sum of the accumulated values ​​calculated in the input layer La. The intermediate layer Lb then outputs the result of the convolution operation to the output layer Lc.

[0098] The output layer Lc calculates the sum of the results of the convolution operations input from the intermediate layer Lb. The result calculated by the output layer Lc represents the product determined to correspond to the product appearance image input to the product recognition model 223. Alternatively, the output value of the output layer Lc can be a vector whose elements represent the probability of similarity between each product (for example, a Q-dimensional vector representing the similarity between product A, product B, ..., and product Q). In this case, the number of dimensions is the number of products to be recognized.

[0099] Furthermore, the values ​​of the coefficients to be accumulated in the output of the input layer La and the values ​​of the coefficients to be accumulated in the output of the intermediate layer Lb are determined by repeatedly performing supervised learning.

[0100] The neural network structure of the product recognition model 223 is not limited to a three-layer structure. The product recognition model 223 may be a model other than a neural network, such as a template storing images of the appearance of multiple products taken from different directions.

[0101] (Structure of deformation and discoloration judgment model)

[0102] Using Figure 6 to explain the structure of the deformation discoloration determination model 224. Figure 6 is a diagram showing an example of the structure of the deformation discoloration determination model stored in the store server.

[0103] The deformation discoloration determination model 224 is, for example, a model described using a publicly known neural network. The deformation discoloration determination model 224 is a model that learns the degree of deformation and the degree of discoloration in a large number of appearance images of various products by performing supervised learning (machine learning) in advance. Figure 6 The example illustrated is an example in which the deformation discoloration determination model 224 is configured as a neural network having a three-layer structure of an input layer Ld, an intermediate layer Le, and an output layer Lf.

[0104] The input layer Ld has a plurality of nodes, and a plurality of different feature amounts extracted from the appearance image of the product and a plurality of different feature amounts extracted from the ideal appearance image of the product are respectively input to each node. The input layer Ld outputs a cumulative value obtained by multiplying a coefficient prescribed for each of the plurality of different feature amounts to the cumulative value to the intermediate layer Le.

[0105] The intermediate layer Le, also referred to as a convolution layer, has a plurality of nodes, and a so-called convolution operation is performed at each node to calculate the sum of the cumulative values calculated in the input layer Ld. Then, the intermediate layer Le outputs the result of the convolution operation to the output layer Lf.

[0106] The output layer Lf calculates the sum of the results of the convolution operation input from the intermediate layer Le. The sum calculated by the output layer Lf indicates the degree of deformation of the product input to the deformation discoloration determination model 224 and the degree of discoloration of the appearance color of the product.

[0107] In addition, the neural network configured by the deformation discoloration determination model 224 is not limited to a three-layer structure. Furthermore, the deformation discoloration determination model 224 can be a model in a form other than a neural network, and can be, for example, a template in which appearance images of a plurality of products taken from different directions are stored.

[0108] Furthermore, in the present embodiment, one model is used to determine the degree of deformation of the product and the degree of discoloration of the appearance color of the product, but the degree of deformation and the degree of discoloration of the appearance color can be determined using other models.

[0109] (Data structure of sales file)

[0110] Using Figure 7 to explain the data structure of the sales file 225. Figure 7 is a diagram showing an example of the data structure of the sales file stored in the store server.

[0111] The sales file 225 is a file in which the product registration file 422 output from the mobile terminal 40 in association with the transaction ID (see FIG. 19) is stored in association with the transaction ID.Figure 8 ), settlement file 423 (see Figure 8 ) and the appearance image of the product output from the mobile terminal 40 to generate the sales file 225.

[0112] The sales file 225 stores sales information indicating the contents of the sales in association with the transaction ID. The sales information includes, for example, the settlement date and time, the product ID, the product name, the normal price, the sales price and the appearance image of the product.

[0113] The settlement date and time is the date and time when the settlement is made by the mobile terminal 40. In addition, the settlement method can be stored together with the settlement date and time.

[0114] The product ID is the product code of the purchased product.

[0115] The product name is the name of the product associated with the product code indicated by the product ID.

[0116] The normal price is the normal price of the product associated with the product code indicated by the product ID.

[0117] The sales price is the sales price of the corresponding product. The sales price is the amount obtained by subtracting the discount amount from the normal price of the product.

[0118] The appearance image of the product is the appearance image of the product taken by the camera 443 of the mobile terminal 40. The appearance image of the product serves as the basis for indicating that the sales price of the product is the appropriate sales price.

[0119] In addition, the information stored in the sales file 225 is not limited to the information shown in Figure 7 For example, the customer ID can be stored in the sales file 225 in the case where the customer has been registered as a member.

[0120] (Hardware structure of mobile terminal)

[0121] The hardware configuration of the mobile terminal 40 will be described using Figure 8 is a hardware block diagram showing an example of the hardware structure of the mobile terminal included in the sales price prompting system. Figure 8

[0122] The mobile terminal 40 includes a structure in which the control section 41, the storage section 42, the peripheral device controller 44 and the communication controller 45 are connected to each other via the internal bus 43.

[0123] ​The control section 41 controls the operation of the mobile terminal 40 as a whole. The control section 41 includes a CPU 411, a ROM 412, and a RAM 413. The CPU 411 is connected to the ROM 412 and the RAM 413 via an internal bus of an address bus, a data bus, and the like. The CPU 411 loads various programs stored in the ROM 412, the storage section 42, into the RAM 413. The CPU 411 controls the operation of the mobile terminal 40 by operating in accordance with the various programs loaded in the RAM 413. That is, the control section 41 has a general-purpose computer structure. In addition, the configuration of the control section 41 is not limited to this, and for example, the CPU 411 and the ROM 412, the RAM 413 can be directly connected without the internal bus 43.

[0124] The storage section 42 is a storage device such as an HDD, an SSD, or the like. Furthermore, the storage section 42 can be a nonvolatile memory such as a flash memory that retains stored information even when the power is turned off. The storage section 42 stores a control program 421, a product registration file 422, and a settlement file 423.

[0125] The control program 421 is a program that controls the operation of the mobile terminal 40 as a whole. The control program 421 can be provided in a state of being stored in the storage section 42, or can be stored in a computer-readable non-transitory storage medium such as a CD-ROM, a floppy disk (FD), a CD-R, a DVD, or the like in an installable form or an executable form and provided. Furthermore, the control program 421 can be provided by being stored in a computer connected to a network and downloaded via the network. Also, the control program 421 can be provided or distributed via a network such as the Internet.

[0126] The product registration file 422 is a file in which registered product information is stored in association with a transaction code (hereinafter referred to as a transaction ID) that specifies a transaction. In addition, specifically, the product ID, the product category, the product name, the normal price, and the sale price received from the store server 20 are stored, and the appearance image of the registered product is stored. The product registration file 422 can include the number of products registered as needed or the like.

[0127] The settlement file 423 is a file in which settlement contents (for example, settlement date and time, settlement method, settlement contents, settlement amount, and the like) related to a transaction are stored in association with the transaction ID.

[0128] Furthermore, the control section 41 has a display device 441, an operation device 442, and a camera 443 via which input and output of various information is performed by a peripheral controller 44.

[0129] The display device 441 is, for example, a display constituted by a liquid crystal panel or an organic EL panel. The display device 441 displays various image information, text information, and the like related to a transaction.

[0130] The operation device 442 is, for example, a touch panel stacked on the display device 441. The operation device 442 acquires information on an operator's operation of the mobile terminal 40 and outputs the information to the control unit 41.

[0131] The camera 443 is an imaging device having a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and captures the direction it is facing. The camera 443 is provided on the back side of the display device 441 of the mobile terminal 40 .

[0132] Furthermore, the control unit 41 performs various communications with the store server 20 via the communication controller 45. Specifically, the mobile terminal 40 outputs an image of the appearance of a product captured by the camera 443 to the store server 20. Furthermore, the mobile terminal 40 obtains the price of the product captured in the image from the store server 20. Furthermore, the mobile terminal 40 outputs a product registration file 422 and a settlement file 423 to the store server 20.

[0133] (Example of information displayed on a mobile terminal)

[0134] Will use Figure 9 and Figure 10 An example of information displayed on the display device 441 of the mobile terminal 40 will be described. Figure 9 This figure shows a state where an image of a product is captured using a mobile terminal. Figure 10 FIG. 1 is a diagram showing a first example of displaying the price of a photographed product on a mobile terminal.

[0135] When a customer finds a desired product Ma in a store, he or she removes the mobile terminal 40 from the shopping cart 15, points the camera 443 toward the desired product Ma, and clicks a start button (not shown) of the camera 443. When the start button of the camera 443 is clicked, the mobile terminal 40 displays the image captured by the camera 443 on the display device 441.

[0136] Next, the customer clicks a photography button 69 (not shown) displayed on the display device 441. When the photography button 69 is clicked, the mobile terminal 40 captures an image of the product Ma displayed on the display device 441 at that time.

[0137] The mobile terminal 40 outputs the captured image of the product Ma to the store server 20. The store server 20 performs the product identification process of specifying the product Ma when the image of the product Ma is acquired from the mobile terminal 40. The store server 20 reads the price of the product Ma from the product master file 222 when the product Ma is specified. Then, the store server 20 outputs the price of the product Ma to the mobile terminal 40.

[0138] The mobile terminal 40 superimposes the price of the product Ma on the image of the product Ma on the display device 441 and displays it when the price of the product Ma is acquired from the store server 20.

[0139] Figure 10 An example of a state in which the price of the product Ma is displayed on the mobile terminal 40 is shown.

[0140] At this time, the price display 61, the registration button 65, the checkout button 66, and the return button 68 are superimposed on the image of the product Ma and displayed on the display device 441.

[0141] The price display 61 is a display showing the price of the product Ma acquired from the store server 20.

[0142] The registration button 65 is an operation control that is clicked when the product Ma photographed on the registration screen is registered. When the registration button 65 is clicked, the mobile terminal 40 registers the product Ma in the product registration file 422 (see Figure 8 ).

[0143] The checkout button 66 is an operation control that is clicked when the settlement related to the registered product is performed. When the checkout button 66 is clicked, the mobile terminal 40 performs the settlement process related to the registration content of the product registration file 422.

[0144] The return button 68 is an operation control that is clicked when the user does not want to purchase the product Ma. When the return button 68 is clicked, the mobile terminal 40 transitions to a state in which the image photographed by the camera 443 is displayed on the display device 441 (the state in Figure 9 ). That is, the image photographed by the camera 443 of the mobile terminal 40 is displayed on the display device 441. The state before the photographing button 69 is clicked is transitioned.

[0145] (Example of the price of the product displayed on the mobile device)

[0146] An example of the price of the product displayed on the display device 441 of the mobile terminal 40 will be described using Figures 11 to 13 . Figure 11 is a diagram showing a second example in which the price of the photographed product is displayed on the mobile terminal. Figure 12 is a diagram showing a third example in which the price of the photographed product is displayed on the mobile terminal.Figure 13 is a view showing a state in which the price of the photographed product Ma is displayed on the mobile terminal 40.

[0147] Figure 11 An example of a state in which the price of the product Ma reflecting the discount based on the time-limited price reduction is displayed on the mobile terminal 40 is shown.

[0148] At this time, on the display device 441, the price display 61, the time-limited price reduction display 62, the registration button 65, the checkout button 66, and the return button 68 are superimposed on the image of the product Ma and displayed.

[0149] The price display 61, the registration button 65, the checkout button 66, and the return button 68 are as described above (see Figure 10 ).

[0150] The time-limited price reduction display 62 is a display showing that the price displayed on the price display 61 is a price reflecting the discount based on the time-limited price reduction.

[0151] Figure 12 An example of a state in which the price of the product Ma reflecting the discount corresponding to the discoloration degree of the product is displayed on the mobile terminal 40 is shown.

[0152] At this time, on the display device 441, the price display 61, the discoloration display 63, the registration button 65, the checkout button 66, and the return button 68 are superimposed on the image of the product Ma and displayed.

[0153] The price display 61, the registration button 65, the checkout button 66, and the return button 68 are as described above (see Figure 10 ).

[0154] The discoloration display 63 is a display indicating that the price displayed on the price display 61 is a price reflecting the discount based on the discoloration of the product. For example, the discoloration display 63 can display "Since the discoloration degree is 5%, a discount amount of 5% is applied". In other words, the discoloration degree and the discount amount corresponding thereto can be displayed. The discount amount can be an amount of money. In addition, for a product whose discoloration degree is equal to or more than a prescribed discoloration degree (for example, a discoloration degree of 21%), it can be displayed that "Since the discoloration degree is large, the product cannot be sold. Please select another product". The registration of the product is prohibited.

[0155] Figure 13 An example of a state in which the price of the product Mb reflecting the discount corresponding to the deformation of the product is displayed on the mobile terminal 40 is shown.

[0156] At this time, on the display device 441, the price display 61, the deformation display 64, the registration button 65, the checkout button 66, and the return button 68 are superimposed on the image of the product Mb and displayed.

[0157] The price display 61, the register button 65, the checkout button 66 and the back button 68 are as described above (see Figure 10 ).

[0158] Deformation display 64 indicates that the price displayed on price display 61 reflects a discount based on the deformation of the product. For example, deformation display 64 may display "Since the degree of deformation is 5%, a discount of 5% will be applied." In other words, the degree of deformation and the corresponding discount amount may be displayed. The discount amount may be a monetary amount. In addition, for products with a degree of deformation exceeding a specified degree (e.g., a degree of deformation of 21%), a message may be displayed saying "Unable to sell due to the large degree of deformation. Please select another product." Registration of this product is prohibited.

[0159] In addition, depending on the status of the product, any of a plurality of discounts may be applied, such as a discount based on a limited-time price reduction, a discount based on a color change, and a discount based on a deformation. In this case, the display device 441 of the mobile terminal 40 displays a corresponding plurality of the limited-time price reduction display 62, the color change display 63, and the deformation display 64.

[0160] (Functional structure of sales price notification system)

[0161] use Figure 14 The functional structure of the sales price presentation system 10 will be described. Figure 14 This is a functional block diagram showing an example of the functional configuration of the sales price presentation system according to the embodiment.

[0162] The control unit 21 of the store server 20 loads the control program 221 into the RAM 213 and operates it, thereby realizing it as a functional unit. Figure 14 The image acquisition unit 31, the commodity specifying unit 32, the discount amount calculating unit 33, the sales price outputting unit 34, and the sales data acquiring unit 35 are shown. In addition, part or all of these functional units may be realized by dedicated hardware.

[0163] The image acquisition unit 31 acquires an image of a product from the mobile terminal 40 .

[0164] The product specifying unit 32 uniquely specifies the product ID (for example, product code) of the product captured in the image acquired by the image acquiring unit 31 .

[0165] The discount amount calculating section 33 calculates a discount amount based on the normal price of the product based on the appearance shape or appearance color of the product. More specifically, the discount amount calculating section 33 calculates a discount amount based on the normal price of the product related to the time-limited price reduction based on the time at which the image of the product is acquired from the terminal device and the closing time of the store. Further, the discount amount calculating section 33 calculates a discount amount based on the normal price of the product based on the degree of difference in appearance shape or appearance color between the image acquired by the image acquiring section 31 and the image of the appearance of the product in the normal state stored by itself. In addition, the discount amount calculating section 33 preferably sets the discount amount of the product related to the time-limited price reduction to be larger as the closing time of the store is closer. Further, the discount amount calculating section 33 preferably sets the discount amount to be larger as the degree of discoloration of the product is larger or as the degree of deformation of the product is larger. The discount amount calculating section 33 uses a model trained to output the degree of discoloration and the degree of deformation of the product in response to the input of the image acquired by the image acquiring section 31 and the image of the appearance of the product in the normal state, and calculates the discount amount according to the degree of deformation or the degree of discoloration. The discount amount is calculated for the product within a predetermined discount range. In addition, it is also possible to determine the product whose degree of discoloration is equal to or more than a prescribed degree of discoloration or the product whose degree of deformation is equal to or more than a prescribed degree of deformation as being unsalable.

[0166] The sales price output section 34 outputs the sales price obtained by subtracting the discount amount from the normal price to the mobile terminal 40. For example, the product ID, the product category, the product name, the normal price, and the sales price are output.

[0167] In addition, information that is the basis of the discount amount calculated by the discount amount calculating section is output. For example, the degree of discoloration of the product and the discount amount, or the degree of deformation and the discount amount can be output. In addition, information indicating that the product is unsalable can be output for the product whose degree of discoloration is equal to or more than a prescribed degree of discoloration or whose degree of deformation is equal to or more than a prescribed degree of deformation.

[0168] The sales data acquiring section 35 acquires the product registration file 422 and the settlement file 423 from the mobile terminal 40.

[0169] Further, the control section 41 of the mobile terminal 40 loads the control program 421 into the RAM 413 and causes it to operate, thereby functioning as Figure 14 the image output section 52, the sales price acquiring section 53, the sales price prompting section 54, the product registration section 55, and the settlement processing section 56 shown in FIG. 1. In addition, part or all of these functional sections can be realized by dedicated hardware.

[0170] The imaging section 51 images an image containing the appearance of the product.

[0171] The image output section 52 outputs the image imaged by the imaging section 51 to the store server 20.

[0172] The sales price acquisition unit 53 acquires the sales price of the product photographed in the image photographed by the imaging unit 51 from the store server 20. For example, the product ID, the product category, the product name, the normal price, and the sales price are acquired from the store server 20. Also, information that is a basis of the discount amount is acquired. For example, the degree of discoloration and the discount amount of the product, or the degree of deformation and the discount amount can be acquired.

[0173] The sales price presentation unit 54 superimposes and presents the sales price of the product on the image of the product. Also, information that is a basis of the discount amount is presented. For example, the degree of discoloration and the discount amount corresponding thereto are presented. Or, for example, the degree of deformation and the discount amount corresponding thereto are presented. In addition, in a case where information indicating that the product cannot be sold is acquired, presentation indicating that the product cannot be sold can be performed.

[0174] The product registration unit 55 performs product registration processing to register the selected product in the product registration file 422. For example, the product ID, the product category, the product name, the normal price, the sales price, and the appearance image of the selected product are stored in the product registration file 422. The product registration unit 55 performs the product registration processing in response to an operation of the registration button 65. In addition, registration of a product having a degree of discoloration that is equal to or higher than a prescribed degree of discoloration, or a product having a degree of deformation that is equal to or higher than a prescribed degree of deformation is prohibited.

[0175] The settlement processing unit 56 performs settlement processing to pay the amount of money related to the product registered in the product registration file 422. Also, the settlement processing unit 56 generates a settlement file 423 indicating the settlement result when the settlement processing is completed.

[0176] (Flow of processing performed by the sales price presentation system)

[0177] The flow of processing performed by the sales price presentation system 10 will be described using Figure 15 and Figure 16 . Figure 15 is a flowchart indicating an example of the flow of processing performed by the sales price presentation system of the embodiment. Figure 16 is a flowchart indicating an example of the flow of discount amount calculation processing.

[0178] First, the flow of processing performed by the mobile terminal 40 will be described.

[0179] The customer who performs shopping at the store operates the mobile terminal 40 and selects the start of product registration from a main menu (not shown). Accordingly, the mobile terminal 40 is in a state in which product registration can be performed (step S11).

[0180] The imaging unit 51 activates the camera 443 to photograph the image of the product (step S12).

[0181] The imaging section 51 displays the image captured by the camera 443 on the display device 441 (step S13).

[0182] The imaging section 51 detects that the photograph button 69 (not shown) displayed on the display device 441 is clicked, and captures the image of the product Ma displayed on the display device 441 at that time. Then, the image output section 52 outputs the image captured by the imaging section 51 to the store server 20 (step S14).

[0183] The sales price acquisition section 53 acquires the sales price of the product Ma captured in the image captured by the imaging section 51 from the store server 20 (step S15). For example, the product ID, the product category, the product name, the normal price, and the sales price are acquired from the store server 20. Further, if there is a discount, information serving as the basis of the discount amount is acquired. Further, in the case of a product that cannot be sold, information indicating that the product cannot be sold is acquired.

[0184] The sales price presentation section 54 superimposes and presents the sales price of the product Ma acquired from the store server 20 on the image of the product Ma (step S16). Further, if there is a discount, information serving as the basis of the discount amount is displayed. For example, the degree of discoloration and the discount amount corresponding thereto, the degree of deformation and the discount amount corresponding thereto are presented. In addition, in the case where information indicating that the product cannot be sold is acquired, the fact that the product cannot be sold can be presented.

[0185] The product registration section 55 determines whether the product Ma has been registered (step S17). When it is determined that the product Ma has been registered (step S17: YES), the process proceeds to step S18. On the other hand, if it is determined that the product Ma has not been registered (step S17: NO), the process returns to step S12. In addition, the product registration section 55 determines that the product Ma has been registered on the condition that the registration button 65 displayed on the display device 441 is clicked. On the condition that the registration button 65 is not clicked and the return button 68 is clicked, it is determined that the product Ma has not been registered. If the registration button is not operated, the product information such as the acquired product ID, the sales price, and the like is temporarily stored in the storage section 42. In addition, if information indicating that the product cannot be sold is acquired, registration of the product is prohibited, and the process returns to step S12. The customer is caused to select another product (the customer is urged to select another product).

[0186] If it is determined in step S17 that the product Ma has been registered, the product registration section 55 updates the product registration file 422 by additionally registering information related to the product Ma in the product registration file 422 (step S18). For example, the product ID, the product category, the product name, the normal price, the sales price, and the appearance image of the selected product are additionally registered in the product registration file 422.

[0187] Subsequently, the settlement processing section 56 determines whether the checkout button 66 displayed on the display device 441 has been clicked (step S19). When it is determined that the checkout button 66 has been clicked (step S19: YES), the process proceeds to step S20. On the other hand, if it is determined that the checkout button 66 has not been clicked (step S19: NO), the process returns to step S12.

[0188] In step S19, if it is determined that the checkout button 66 has been clicked, the settlement processing section 56 performs settlement processing related to the contents registered in the product registration file 422 (step S20). In addition, the settlement method is selected by the customer operating the mobile terminal 40.

[0189] The settlement processing section 56 outputs a settlement file 423 indicating the settlement result to the store server 20 (step S21). In addition, at this time, the product registration file 422 is also output to the store server 20. After that, the mobile terminal 40 ends the process.

[0190] In addition, in the case where it is determined that no product Ma is registered (step S17: NO), it is considered that the customer selects another product of the same product type. For example, a banana Ma that has not been colored is not registered, and another banana Mb that has been colored is captured by the imaging section 51. It is assumed that the customer registers after confirming the sales price of the different product of the same product type. The same applies to the deformed product. Therefore, the sales price prompting section 54 can determine whether it is the same product type as the product prompted last time and change the form of the prompt. The details are as follows.

[0191] The sales price prompting section 54 determines whether the product ID and the sales price are stored in the storage section 42 in the process of S16 (determines whether the product ID and the sales price of the unregistered product are stored) when the sales price is acquired. If it is stored, the sales price acquisition section 53 compares the product ID acquired this time with the product ID (the product ID acquired last time) stored in the storage section 42. If the product IDs are the same (for example, the product ID indicating a banana), the sales price prompting section 54 prompts both the sales price acquired last time and the sales price acquired this time in the storage section 42. In addition, the sales price prompted is the two prices of the last price and the this time price. Even if a plurality of sales prices are prompted, the customer does not know which sales price is the sales price of which product. Therefore, the two sales prices of the sales price acquired last time and the sales price acquired this time are prompted. If the product IDs are different, the storage section 42 is cleared and the acquired sales price is prompted. The process in this case is as described above.

[0192] Further, the sales price of the product registered by the product registration section 55 is not presented. Even if the sales price of the product having the same product ID as the product ID of the product registered by the product registration section 55 last time is acquired, the sales price last time is not presented. That is, when the registration button 65 is operated, the product registration section 55 performs the product registration processing and clears the storage section 42. If the sales price of the product having the same product ID as the product registered last time is acquired, if the sales price of the product registered last time and the product acquired this time are presented, the customer can compare the prices and request cancellation or return of the product registered last time. This situation is prevented. On the other hand, when the registration button 65 is not operated (when the return button 68 is operated), the product ID and the sales price are stored in the storage section 42. In the storage section 42, the product ID (the product ID of the product for which the product registration processing is not performed) and the sales price at the time when the registration button 65 is not operated are stored.

[0193] Next, the flow of the processing performed by the store server 20 will be described.

[0194] The image acquisition section 31 acquires the image in which the product Ma is photographed from the mobile terminal 40 (step S31).

[0195] The product specification section 32 uniquely specifies the product code of the product Ma photographed in the image from the image acquired by the image acquisition section 31 (step S32).

[0196] The discount amount calculation section 33 performs the discount amount calculation processing to calculate the discount amount based on the normal price of the product Ma photographed in the image (step S33). Further, the detailed flow of the discount amount calculation processing performed in step S33 will be described later (see Figure 16 ).

[0197] The sales price output section 34 outputs the sales price obtained by subtracting the discount amount from the normal price to the mobile terminal 40 (step S34). The product information to be output is, for example, the product ID, the product category, the product name, the normal price, and the sales price. Further, if there is a discount, information serving as the basis of the discount amount is output. Further, if it is a product that cannot be sold, information indicating that the product cannot be sold is output.

[0198] The sales data acquisition section 35 acquires information indicating the settlement content (for example, the product registration file 422 and the settlement file 423) from the mobile terminal 40 (step S35).

[0199] The sales data acquisition section 35 updates the sales file 225 based on the information indicating the settlement content acquired in step S35 (step S36). After that, the store server 20 ends the processing.

[0200] Next, the flow of the processing performed by the store server 20 will be described. Figure 16to explain the flow of the discount amount calculation processing performed by the store server 20 in step S33. Figure 15

[0201] The discount amount calculation section 33 determines whether the time at which the image of the product Ma was captured by the image taken by the mobile terminal 40 is a time period of the time-limited price reduction of the store (step S41). If it is determined that it is the time period of the time-limited price reduction (step S41: YES), the process proceeds to step S42. On the other hand, if it is determined that it is not the time period of the time-limited price reduction (step S41: NO), the process proceeds to step S44. In addition, the discount amount calculation section 33 can determine whether the time at which the image was captured is the time period of the time-limited price reduction of the store at the time of output of the image of the product Ma output by the mobile terminal 40.

[0202] In step S41, when it is determined that it is the time period of the time-limited price reduction, the discount amount calculation section 33 determines whether the specified product Ma is an object product of the time-limited price reduction (step S42). If it is determined that the product Ma is the object product of the time-limited price reduction (step S42: YES), the process proceeds to step S43. On the other hand, if it is determined that the product Ma is not the object product of the time-limited price reduction (step S42: NO), the process proceeds to step S44. In addition, the discount amount calculation section 33 determines whether the product Ma is the object product of the time-limited price reduction by referring to the product master file 222.

[0203] In step S42, if it is determined that the product Ma is the object product of the time-limited price reduction, the discount amount calculation section 33 adds the discount corresponding to the time-limited price reduction registered in association with the product Ma in the product master file 222 (step S43). In addition, the discount amount based on the time-limited price reduction is set to be higher as it is closer to the store closing time.

[0204] Next, the discount amount calculation section 33 analyzes the image of the product Ma and determines whether the product Ma has a deformation (step S44). If it is determined that the product Ma has a deformation (step S44: YES), the process proceeds to step S45. On the other hand, if it is determined that the product Ma has no deformation (step S44: NO), the process proceeds to step S46. In addition, the discount amount calculation section 33 determines whether the product Ma has a deformation by the deformation and discoloration determination model 224.

[0205] ​If, in step S44, it is determined that the product Ma is deformed, the discount calculation unit 33 adds the discount associated with the product Ma and corresponding to the degree of deformation to the product master file 222 (step S45). A discount is applied within the applicable discount range based on the degree of deformation. Furthermore, information indicating that products with a degree of deformation exceeding a specified level cannot be sold may be stored. If a discount based on a limited-time price reduction has already been applied in step S43, the discount based on the limited-time price reduction and the discount based on the deformation are added together.

[0206] Next, the discount amount calculation unit 33 analyzes the image of the product Ma and determines whether the product Ma has changed color (step S46). If the product Ma is determined to have changed color (step S46: Yes), the process proceeds to step S47. On the other hand, if the product Ma is determined to have not changed color (step S46: No), the process proceeds to step S48. Furthermore, the discount amount calculation unit 33 uses the deformed color change determination model 224 to determine whether the product Ma has changed color.

[0207] If, in step S46, it is determined that the product Ma has discolored, the discount calculation unit 33 adds the discount associated with the product Ma and corresponding to the degree of discoloration to the product master file 222 (step S47). A discount is applied within the applicable discount range based on the degree of discoloration. Furthermore, information indicating that products with discoloration exceeding a specified level cannot be sold may be stored. In this case, if the discount based on the limited-time price reduction and the discount based on the deformation have already been added together, the discount based on the discoloration is further added.

[0208] The discount amount calculation unit 33 determines the discount amount of the product Ma by adding the discount corresponding to the limited-time price reduction, the discount corresponding to the degree of deformation, and the discount corresponding to the color change (step S48). Thereafter, the process returns to Figure 15 In step S34. In addition, when all of the discounts corresponding to the limited-time price reduction, the discount corresponding to the degree of deformation, and the discount corresponding to the color change are applied, the discount amount may become larger, so it is preferable to pre-set the maximum value of the discount amount (for example, the maximum discount amount is 60% of the product price, etc.).

[0209] The sales price presentation system may have the following configuration.

[0210] The sales price output section outputs the product ID, product category, product name, regular price, and sales price.

[0211] The commodity registration unit performs commodity registration processing when the registration button 65 is operated and clears the storage unit 42 storing the unregistered commodity ID and sales price. When the registration button 65 is not operated, the unregistered commodity ID and sales price are stored in the storage unit 42.

[0212] The sales price presentation section 54 compares the acquired product ID with the product ID stored in the storage section 42, and if they are the same product ID, presents both the previous sales price in the storage section 42 and the sales price acquired this time. If they are different product IDs, the acquired sales price is presented. Accordingly, in the case where the product not registered the previous time is the same product as the product photographed by the imaging section this time, the sales prices can be compared.

[0213] (EFFECTS OF THE EMBODIMENTS)

[0214] As explained above, the sales price presentation system 10 of the present embodiment includes a mobile terminal 40 (terminal device) including an imaging section 51 that photographs an image including the appearance of a product, an image output section 52 that outputs the image photographed by the imaging section 51 to a store server 20 (information processing device), a sales price acquisition section 53 that acquires the sales price of the product photographed in the image from the store server 20, and a sales price presentation section 54 that superimposes the sales price of the product on the image and presents it, and the store server 20 including an image acquisition section 31 that acquires the image from the mobile terminal 40, a product specification section 32 that uniquely specifies the product from the image acquired by the image acquisition section 31, a discount amount calculation section 33 that calculates a discount amount based on the normal price of the product based on the shape or color of the appearance of the product, and a sales price output section 34 that outputs the sales price obtained by subtracting the discount amount from the normal price to the mobile terminal 40. Therefore, the sales price can be presented according to the state of the product that the customer wants to purchase. Further, since the salesclerk does not need to attach a discount tag indicating the discount amount to each product, the work load of the salesclerk can be reduced.

[0215] Further, in the sales price presentation system 10 of the present embodiment, the discount amount calculation section 33 further calculates the discount amount based on the normal price of the product according to the time when the image is acquired from the mobile terminal 40 (terminal device) and the closing time of the store. Therefore, without the need to attach a tag or the like indicating the discount amount of the product on which the time-limited price reduction is performed to each product, the work load of the salesclerk can be reduced.

[0216] Further, in the sales price presentation system 10 of the present embodiment, the sales price output section 34 further outputs information that is the basis of the discount amount calculated by the discount amount calculation section 33 to the mobile terminal 40 (terminal device), and the sales price presentation section 54 further superimposes and presents the information that is the basis of the discount amount on the image. Therefore, the customer can easily understand the reason for setting the discount amount.

[0217] Furthermore, in the sales price display system 10 of this embodiment, the discount amount calculation unit 33 uses a deformation and discoloration determination model 224 trained to output the degree of discoloration and deformation of the product, based on the image acquired by the image acquisition unit 31 and the product's appearance image in a normal state, to calculate a discount amount corresponding to the degree of deformation or discoloration of the product. This makes it easy to calculate a discount amount based on the discoloration or deformation of the product.

[0218] Furthermore, the store server 20 (information processing device) of this embodiment includes an image acquisition unit 31 that acquires an image including the appearance of a product from a mobile terminal 40 (terminal device), a product identification unit 32 that uniquely identifies the product based on the image acquired by the image acquisition unit 31, a discount amount calculation unit 33 that calculates a discount amount from the regular price of the product based on the appearance shape or appearance color of the product, and a sales price output unit 34 that outputs the price obtained by subtracting the discount amount from the regular price to the mobile terminal 40. Therefore, the sales price can be set according to the conditions of the product that the customer wishes to purchase.

[0219] While the embodiments of the present invention have been described above, these embodiments are provided as examples and are not intended to limit the scope of the invention. The novel embodiments may be implemented in various other forms and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. These embodiments and their variations are intended to be within the scope and spirit of the invention and are within the scope of the invention as set forth in the claims and their equivalents.

Claims

1. A sales price presentation system including a terminal device and an information processing device, characterized by: the terminal device including: an imaging section that captures an image including an appearance of a product; an image output section that outputs the image captured by the imaging section to the information processing device; a sales price acquisition section that acquires a sales price of the product captured in the image from the information processing device; and a sales price presentation section that presents the sales price of the product superimposed on the image, the information processing device including: an image acquisition section that acquires the image from the terminal device; a product specification section that uniquely specifies the product based on the image acquired by the image acquisition section; a discount amount calculation section that calculates a discount amount based on a normal price of the product based on an appearance shape or an appearance color of the product; and a sales price output section that outputs, to the terminal device, a sales price obtained by subtracting the discount amount from the normal price.

2. The sales price presentation system according to claim 1, wherein the discount amount calculation section further calculates a discount amount based on a normal price of the product based on a time at which the image is acquired from the terminal device and a closing time of a store.

3. The sales price presentation system according to claim 1 or 2, wherein the sales price output section further outputs, to the terminal device, information that is a basis of the discount amount calculated by the discount amount calculation section, and the sales price presentation section further presents the information that is the basis of the discount amount superimposed on the image.

4. The sales price presentation system according to claim 1, wherein the discount amount calculation section uses a model trained to output a degree of discoloration and a degree of deformation of the product corresponding to an input of an image acquired by the image acquisition section and an appearance of the product in a normal state, and calculates a discount amount based on the degree of deformation or the degree of discoloration of the product.

5. The sales price presentation system according to claim 1, wherein the discount amount calculation section calculates a discount amount for a product within a predetermined discountable range.

6. The sales price presentation system according to claim 1, wherein the terminal device further includes: a product registration section that performs product registration processing of the product presented by the sales price presentation section based on a case where a registration button is operated.

7. The sales price presentation system according to claim 1, wherein the discount amount calculation section uses a model trained to output a degree of discoloration and a degree of deformation of the product corresponding to an input of an image acquired by the image acquisition section and an appearance of the product in a normal state, and calculates a discount amount based on the degree of deformation or the degree of discoloration of the product, the sales price output section outputs, to the terminal device, the degree of discoloration and the discount amount or the degree of deformation and the discount amount of the product, and the sales price presentation section further presents the degree of discoloration and the discount amount corresponding thereto or the degree of deformation and the discount amount corresponding thereto superimposed on the image.

8. The sales price presentation system according to claim 7, wherein ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The sales price output section outputs information indicating that the product cannot be sold, with respect to a product whose discoloration degree is equal to or greater than a prescribed discoloration degree, or a product whose deformation degree is equal to or greater than a prescribed deformation degree, The sales price prompting section prompts a case where the product cannot be sold, in a case where information indicating that the sales cannot be performed is acquired.

9. An information processing apparatus, characterized by comprising: Including: An image acquisition section acquires an image including an appearance of a product from a terminal device; A product specification section uniquely specifies the product based on the image acquired by the image acquisition section; A discount amount calculation section calculates a discount amount based on a normal price of the product, based on an appearance shape or an appearance color of the product; And A sales price output section outputs a price obtained by subtracting the discount amount from the normal price, to the terminal device.

10. A storage medium storing a program that causes a computer that controls a sales price prompting system including a terminal device and an information processing device to control the terminal device and the information processing device to function as each of the following sections: Causes the terminal device to function as each of the following sections: An image capturing section that captures an image including an appearance of a product; An image output section that outputs the image captured by the image capturing section to an information processing device; and The sales price acquisition unit acquires the sales price of the product photographed in the image from the information processing device. A sales price prompting section that prompts a sales price of the product superimposed on the image, Causes the information processing device to function as each of the following sections including: An image acquisition section that acquires the image from the terminal device; A product specification section that uniquely specifies the product based on the image acquired by the image acquisition section; A discount amount calculation section that calculates a discount amount based on a normal price of the product, based on an appearance shape or an appearance color of the product; and A sales price output section that outputs a sales price obtained by subtracting the discount amount from the normal price, to the terminal device. ​

Images