Information processing device, information processing method, and information processing program

Abstract

To provide a technique for providing an incentive, such as flexible discount of a commodity.SOLUTION: An information processing apparatus includes a first acquisition unit, a second acquisition unit, and a display control unit. The first acquisition unit acquires a production time and date of a commodity on the basis of an image related to the commodity captured by an imaging unit. The second acquisition unit acquires incentive information of the commodity based on the production time and date. The display control unit causes a display unit to display a proper object including the incentive information.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In commercial facilities such as supermarkets, the introduction of a POS system in which consumers register products themselves using smartphones or the like is progressing. Such a system is an effective system for labor saving and contactless shopping, and there is a possibility that its introduction will accelerate in the future.

[0003] On the other hand, along with the labor saving in commercial facilities, there is a demand for improving the efficiency of work in the sales floor such as attaching discount stickers and replacing price cards.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by the embodiments of the present invention is to provide a technique for realizing the provision of incentives such as discounts on flexible products or services by utilizing the functions of mobile terminals such as smartphones.

Means for Solving the Problems

[0006] In one embodiment, the information processing apparatus includes a first acquisition unit, a second acquisition unit, and a display control unit. The first acquisition unit acquires the manufacturing date and time of the product based on an image related to the product photographed by the photographing unit. The second acquisition unit acquires the incentive information of the product based on the manufacturing date and time. The display control unit displays the main object including the incentive information on the display unit. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a block diagram illustrating an information processing system according to the first embodiment. [Figure 2] Figure 2 is a diagram illustrating the data structure of product data according to the first embodiment. [Figure 3] Figure 3 is a schematic diagram showing an example of an AR image according to the first embodiment. [Figure 4] Figure 4 is a schematic diagram showing an example of the screen display of a user terminal according to the first embodiment. [Figure 5] Figure 5 is a schematic diagram showing another example of the screen display of a user terminal according to the first embodiment. [Figure 6] Figure 6 is a schematic diagram illustrating yet another example of the screen display of a user terminal according to the first embodiment. [Figure 7] Figure 7 is a flowchart illustrating the information processing procedure by a user terminal according to the first embodiment. [Figure 8] Figure 8 is a block diagram illustrating an information processing system according to the second embodiment. [Figure 9] Figure 9 is a flowchart illustrating the information processing procedure by the server according to the second embodiment. [Modes for carrying out the invention]

[0008] [First Embodiment] Embodiments will be described below with reference to the drawings. In each drawing, the same reference numerals are used for identical components whenever possible, and redundant explanations are omitted.

[0009] (Example configuration) Figure 1 is a block diagram illustrating an information processing system 100 according to the first embodiment. The information processing system 100 includes a server 1 and a user terminal 2. Server 1 and user terminal 2 are connected to each other via a network, allowing for seamless communication. For example, the network consists of one or more networks selected from various networks such as the Internet, a mobile communication network, and a LAN (Local Area Network). The one or more networks may include a wireless network or a wired network.

[0010] Server 1 is an electronic device that collects and processes data. This electronic device includes a computer. Server 1 communicates freely with user terminal 2 via a network. Server 1 receives various data from user terminal 2 and outputs various data to user terminal 2. For example, Server 1 manages product information in stores such as supermarkets. An example configuration of Server 1 will be described later.

[0011] User terminal 2 is an electronic device capable of communicating with other electronic devices. User terminal 2 is, for example, a device used by a customer in a store. For example, user terminal 2 is a smartphone or a tablet device. The term "customer" may be replaced with "user" or "person." User terminal 2 is an example of an information processing device. An example configuration of user terminal 2 will be described later.

[0012] This section describes an example configuration for Server 1. Server 1 is an electronic device that includes a processor 11, main memory 12, auxiliary storage device 13, and communication interface 14. Each component of Server 1 is connected to the others so that signals can be input and output. In Figure 1, the interface is labeled "I / F".

[0013] The processor 11 is the central part of the server 1. The processor 11 is an element that makes up the computer of the server 1. For example, the processor 11 is a CPU (Central Processing Unit), but is not limited to this. The processor 11 may be composed of various circuits. The processor 11 loads a program that is pre-stored in the main memory 12 or auxiliary storage device 13 into the main memory 12. The program is a program that causes the processor 11 of the server 1 to implement the various parts described later and to execute the functions of each part. The processor 11 performs various operations by executing the program loaded into the main memory 12.

[0014] Main memory 12 corresponds to the main memory portion of server 1. Main memory 12 is an element that constitutes the computer of server 1. Main memory 12 includes non-volatile memory areas and volatile memory areas. In the non-volatile memory area of ​​main memory 12, the operating system or programs are stored. Main memory 12 uses the volatile memory area as a work area where data is rewritten as needed by the processor 11. For example, main memory 12 includes ROM (Read Only Memory) as a non-volatile memory area. For example, main memory 12 includes RAM (Random Access Memory) as a volatile memory area.

[0015] The auxiliary storage device 13 corresponds to the auxiliary storage portion of server 1. The auxiliary storage device 13 is an element that constitutes the computer of server 1. The auxiliary storage device 13 is an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), HDD (Hard Disc Drive), or SSD (Solid State Drive), etc. The auxiliary storage device 13 stores the above-mentioned program, data used by the processor 11 in performing various processes, and data generated by the processing performed by the processor 11.

[0016] The auxiliary memory device 13 includes a product data storage area 131. The product data storage area 131 stores product data regarding at least one product. The product data indicates data associated with a product code, elapsed time, incentive information, etc. The product code is unique identification information assigned to each product for individually identifying the product. The elapsed time indicates the elapsed time from the manufacturing date and time of the product. Each of the product codes is associated with one or more elapsed times. For example, for the product with the product code "XXXXX", information such as "less than 9 hours", "9 hours or more", "10 hours or more" of elapsed time is associated. The incentive information includes discount information for the product according to the elapsed time. Each of the incentive information is associated with the elapsed time. Different information can be set for each elapsed time for the incentive information. For example, for the elapsed time of "9 hours or more", incentive information of "10% off" may be associated. For the elapsed time of "10 hours or more", incentive information of "30% off" may be associated. The incentive information for a certain elapsed time may be different for each product. For example, for the product with the product code "XXXXX", for the elapsed time of "9 hours or more", incentive information of "10% off" may be associated. On the other hand, for the product with the product code "YYYYY", for the elapsed time of "9 hours or more", incentive information of "50% off" may be associated. The incentive information for the elapsed time may be set in advance, or may be appropriately set or updated by a store manager or the like. The product data can be updated each time the incentive information is set or updated.

[0017] The auxiliary storage device 13 includes a registered product data storage area 132. The registered product data storage area 132 stores registered product data. The registered product data includes information about purchased products registered by customers. The purchased products are also referred to as registered products. The registered product data indicates data in which a registered product code, incentive information, an amount, etc. are associated. The registered product code is unique identification information assigned for each registered product in order to individually identify the registered products registered by customers. The incentive information indicates, for each registered product, incentive information according to the time when the product was registered by the customer. For example, when a product with a product code "XXXXX" is registered by a customer at a point "9 hours or more" from the manufacturing date and time, the incentive information for the product with the registered product code "XXXXX" indicates "10% off". The amount indicates the price before the incentive is applied, set for each registered product. The amount indicates the amount managed for each product with the registered product code as a key by a product master (not shown). The registered product data can be updated each time a product is registered by a customer.

[0018] The communication interface 14 includes various interfaces that connect the server 1 to other electronic devices via a network so as to be communicable according to a predetermined communication protocol.

[0019] Note that the hardware configuration of the server 1 is not limited to the above-described configuration. The server 1 can appropriately omit and change the above-described components and add new components.

[0020] A configuration example of the user terminal 2 will be described. The user terminal 2 is an electronic device including a processor 21, a main memory 22, an auxiliary storage device 23, a communication interface 24, a display device 25, a speaker 26, an input device 27, and a camera 28. Each part constituting the user terminal 2 is connected so as to be able to input and output signals to and from each other.

[0021] The processor 21 is the central part of the user terminal 2. The processor 21 is an element that constitutes the computer of the user terminal 2. The processor 21 has the same hardware configuration as the processor 11 described above. The processor 21 performs various operations by executing programs that are pre-stored in the main memory 22 or auxiliary storage device 23. The programs cause the processor 21 of the user terminal 2 to implement the parts described later and to execute the functions of each part.

[0022] Main memory 22 corresponds to the main memory portion of user terminal 2. Main memory 22 is an element that constitutes the computer of user terminal 2. Main memory 22 has the same hardware configuration as main memory 12 described above. Main memory 22 stores programs.

[0023] The auxiliary storage device 23 corresponds to the auxiliary storage portion of the user terminal 2. The auxiliary storage device 23 is an element that constitutes the computer of the user terminal 2. The auxiliary storage device 23 has the same hardware configuration as the auxiliary storage device 13 described above. The auxiliary storage device 23 stores the program described above.

[0024] The auxiliary storage device 23 stores user information of user terminal 2. User information includes user identification information, user terminal 2 identification information, etc. User identification information is unique identification information assigned to each user to identify each user individually. User terminal 2 identification information is unique identification information assigned to each user terminal 2 to identify each user terminal 2 individually. User terminal 2 identification information includes terminal-specific identifiers such as the IP address or serial number of user terminal 2.

[0025] The auxiliary storage device 23 includes an object data storage area 230. The object data storage area 230 stores data for at least one temporary object. The temporary object data indicates data for a temporary object associated with AR (Augmented Reality) marker image information. The AR marker image information is information that indicates an image registered in advance as an AR marker. The AR marker image information is information for displaying, for example, at least one of a temporary object indicating that an incentive may be applied to a product, and the actual object described later. The temporary object indicates, for example, an object such as a temporary discount sticker. The temporary discount sticker is, for example, a sticker that displays only the outer frame of a discount sticker. The temporary object does not include incentive information for each product. The AR marker image information may be set in advance, or it may be set or updated as appropriate by an administrator or the like.

[0026] The communication interface 24 includes various interfaces that enable the user terminal 2 to communicate with other devices via a network, in accordance with a predetermined communication protocol.

[0027] The display device 25 is a device capable of displaying various screens under the control of the processor 21. For example, the display device 25 is a liquid crystal display or an EL (electroluminescence) display. The display device 25 is an example of a display unit.

[0028] Speaker 26 is a device that can output sound under the control of user terminal 2.

[0029] The input device 27 is a device capable of inputting data or instructions to the user terminal 2. For example, the input device 27 includes a keyboard or a touch panel.

[0030] Camera 28 is a device capable of acquiring image data within the shooting range. Camera 28 is a device capable of reading various types of data. For example, Camera 28 is a device capable of reading data from captured code symbols. Code symbols include barcode symbols and two-dimensional code symbols such as QR codes (registered trademarks). For example, Camera 28 reads a code symbol attached to a product and registers the product. Camera 28 is a device capable of reading data from AR markers. For example, Camera 28 reads an AR marker attached to a product and displays AR content. Camera 28 is an example of an image capture unit.

[0031] The hardware configuration of user terminal 2 is not limited to the configuration described above. User terminal 2 may, as appropriate, omit or change the above-described components and add new components.

[0032] The various components implemented in the aforementioned processor 21 will now be described.

[0033] The processor 21 implements a first acquisition unit 210, a second acquisition unit 211, a display control unit 212, and a registration unit 213. Each part implemented in the processor 21 can also be called a function. Each part implemented in the processor 21 can also be said to be implemented in the control unit which includes the processor 21 and the main memory 22.

[0034] The first acquisition unit 210 acquires a product code based on an image related to the product captured via the camera 28. The image related to the product includes an AR marker image, a manufacturing date and time image, and a product code image, etc. The AR marker image is an image that has been registered in advance as an AR marker. The AR marker image is an example of an image for displaying at least one of a temporary object and a real object. The manufacturing date and time image is an image that shows the date and time the product was manufactured. The product code image is an image that shows the product code. Images related to the product are also called AR images. For example, the first acquisition unit 210 acquires a product code from an image related to the product in response to an image related to the product being captured via the camera 28. For example, the first acquisition unit 210 performs image processing on the product code image included in the captured image related to the product and acquires the product code of the product. The product code image may also include text information. In the following description, "in response" may be read as "based on".

[0035] The first acquisition unit 210 acquires the manufacturing date and time based on an image related to the product captured via the camera 28. For example, the first acquisition unit 210 acquires the manufacturing date and time from an image related to the product in response to an image related to the product being captured via the camera 28. For example, the first acquisition unit 210 performs image processing on the manufacturing date and time image included in the captured image related to the product to acquire the manufacturing date and time of the product. The manufacturing date and time image may also include text information.

[0036] The first acquisition unit 210 acquires AR marker image information based on images related to the product captured via the camera 28. For example, the first acquisition unit 210 acquires AR marker image information from images related to the product in response to images related to the product being captured via the camera 28. For example, the first acquisition unit 210 acquires AR marker image information by image processing of AR marker images contained in the captured images related to the product. The first acquisition unit 210 acquires information about a temporary object associated with the acquired AR marker image information stored in the auxiliary storage device 23.

[0037] The second acquisition unit 211 acquires product incentive information based on the manufacturing date and time. For example, the second acquisition unit 211 acquires product incentive information based on the product code and the manufacturing date and time. The second acquisition unit 211 outputs the product code to the server 1 via the network. The second acquisition unit 211 outputs the manufacturing date and time to the server 1 via the network. The second acquisition unit 211 may output the product code and the manufacturing date and time to the server 1 simultaneously, or at different times. In response to the output of the product code and manufacturing date and time to the server 1, the second acquisition unit 211 acquires product incentive information from the server 1. In the following description, "acquisition" may be read as "reception". Also, "output" may be read as "transmission".

[0038] Server 1 calculates the elapsed time from the manufacturing date and time to the current date and time based on the manufacturing date and time obtained from the user terminal 2 via the network. The current date and time indicates the date and time when an image related to the product was taken by the user terminal 2 via the camera 28. The current date and time may also be the date and time when at least one of the product code and the manufacturing date and time was obtained by Server 1. Server 1 obtains incentive information associated with the calculated elapsed time for the product code obtained from the user terminal 2 via the network from the product data stored in the product data storage area 131. Server 1 outputs the product incentive information to the user terminal 2 via the network.

[0039] The display control unit 212 displays a temporary object on the display device 25 based on an image related to the product captured via the camera 28. The display control unit 212 displays the temporary object on the display device 25 in response to the acquisition of a temporary object based on the capture of an image related to the product by the first acquisition unit 210.

[0040] The display control unit 212 displays the main object on the display device 25 based on the incentive information acquired by the second acquisition unit 211. The main object is an object that includes incentive information for a product. The main object is an object obtained by applying incentive information to a temporary object. The main object may also be an object obtained by combining or superimposing an object generated based on incentive information onto a temporary object. The main object is, for example, an object that shows a discount sticker. In response to the acquisition of incentive information by the second acquisition unit 211, the display control unit 212 displays the main object on the display device 25. In response to the acquisition of incentive information, the display control unit 212 changes the display of the temporary object to the display of the main object. The display control unit 212 may replace the temporary object with the main object, or it may gradually transition from the temporary object to the main object.

[0041] The display control unit 212 changes the display mode of this object according to the incentive information. For example, the display control unit 212 changes the display mode of this object based on the content of the incentive indicated by the incentive information. The display control unit 212 displays this object when the incentive information indicates that an incentive will be given. The display control unit 212 may display this object indicating that there is no incentive when the incentive information indicates that no incentive will be given, or it may not display this object. For example, if the incentive information indicates an incentive of "20% off", the display control unit 212 displays this object showing a discount sticker indicating "20% off". If the incentive information indicates an incentive of "half price", the display control unit 212 displays this object showing a discount sticker indicating "half price". If the incentive information indicates no incentive, the display control unit 212 may display this object indicating that there is no incentive. This object indicating no incentive is a blank object and an object containing incentive information indicating no incentive, etc. The display control unit 212 may hide a temporary object without displaying this object when the incentive information indicates no incentive. The display control unit 212 displays the main object indicating "sale prohibited" if the incentive information indicates "sale prohibited". If the incentive information cannot be obtained, the display control unit 212 maintains the display of the temporary object until the incentive information is obtained.

[0042] The registration unit 213 registers an incentive corresponding to incentive information to a product based on the user's selection operation of the product. The selection operation refers to an operation in which the user selects a product displayed on the display device 25 by touch operation or the like. For example, the user may select one of the at least one product displayed on the display device 25. In response to the user's selection operation, the registration unit 213 outputs the product code and incentive information associated with the selected product to the server 1 via the network, associating them with the product registration instruction. The registration unit 213 may output the product code and incentive information to the server 1 simultaneously associating them with the product registration instruction, or it may output them at different timings. Product registration instructions provide instructions for registering a product.

[0043] Server 1 updates the registered product data based on the product registration instruction. Server 1 adds a record to the registered product data that includes the product code, incentive information, and price. Server 1 retrieves the price of the product associated with the product code from the product master.

[0044] This section describes an example of product data structure. Figure 2 is a diagram illustrating the data structure of product data according to the first embodiment.

[0045] Product data represents data associated with product code, elapsed time, and incentive information. For example, a product with product code "XXXXX" is associated with elapsed time information for "less than 9 hours," "9 hours or more," "10 hours or more," "11 hours or more," and "12 hours or more." Each of these elapsed time categories is associated with incentive information such as "none," "10% off," "20% off," "30% off," "50% off," and "sale prohibited." Similarly, a product with product code "YYYYY" is associated with multiple elapsed times and incentive information corresponding to each elapsed time. Server 1 updates the product data as needed.

[0046] This section explains an example of image configuration for augmented reality (AR). Figure 3 is a schematic diagram showing an example of an AR image according to the first embodiment.

[0047] AR image A includes AR marker image AA, manufacturing date and time image AB, and product code image AC. AR marker image AA is an image that has been pre-registered as an AR marker. In this case, AR marker image AA shows a rectangular frame. Manufacturing date and time image AB is an image that shows the manufacturing date and time. Manufacturing date and time image AB may be a figure resembling a clock. Manufacturing date and time image AB may also be an image that includes text information. Product code image AB is an image that shows the product code. Product code image AB may be a simplified version of the product code. Product code image AB may also be an image that includes text information. Manufacturing date and time image AB and product code image AC may be displayed in any manner. Manufacturing date and time image AB and product code image AC may be images that show information such as still images, videos, or three-dimensional images. Note that manufacturing date and time image AB can be represented by coloring the area between the hour and minute hands of a clock. Therefore, all images included in AR image A can be printed with existing label printers. Furthermore, manufacturing date and time image AB is a manner in which customers can recognize the manufacturing date and time by coloring the area between the hour and minute hands of a clock. As a result, AR image A also serves as an image that allows customers to recognize the manufacturing date and time, eliminating the need to issue a separate label for customers to recognize the manufacturing date and time.

[0048] This section describes AR image A, which is displayed on the display device 25 of user terminal 2. Figure 4 is a schematic diagram showing an example of the screen display of the user terminal 2 according to the first embodiment. The user of user terminal 2 holds up the AR image A attached to the desired product to the camera 28 while shopping in the store. Figure 4 shows the AR image A captured by the camera 28. The camera 28 can capture at least one AR image A that is included in the captureable area of ​​screen Scr.

[0049] The user holds the AR images Aa and Ab attached to product Pa and product Pb up to the camera 28. The display device 25 displays the AR images Aa and Ab in a recognizable manner.

[0050] This section describes the temporary object displayed on the display device 25 of user terminal 2. Figure 5 is a schematic diagram showing another example of the screen display of the user terminal 2 according to the first embodiment. Figure 5 shows temporary objects Ba and Bb generated based on AR images Aa and Ab captured by camera 28. Temporary objects Ba and Bb represent temporary discount stickers indicating that an incentive may be offered on the product to which AR images Aa and Ab are attached. The temporary discount stickers do not contain specific incentive information and are all displayed in the same manner.

[0051] This section describes the object displayed on the display device 25 of user terminal 2. Figure 6 is a schematic diagram illustrating yet another example of the screen display of the user terminal 2 according to the first embodiment. Figure 6 shows the main object Ca and the main object Cb generated based on AR images Aa and Ab captured by camera 28. The main object Ca and the main object Cb are objects that indicate incentive information. The main object Ca is a discount sticker indicating that the incentive information for product Pa is "10% off". The main object Cb indicates that the incentive information for product Pa is "none". In Figure 6, the main object Cb is shown as a white sticker, but the display control unit 212 may hide the object.

[0052] (Example of operation) The procedure for processing by user terminal 2 will be explained below. In the following explanation focusing on Server 1, you may substitute Server 1 for Processor 11. Similarly, in the explanation focusing on User Terminal 2, you may substitute User Terminal 2 for Processor 21. The processing procedure described below is merely an example, and each process may be modified as much as possible. Furthermore, depending on the embodiment, steps in the processing procedure described below may be omitted, replaced, or added as appropriate.

[0053] This scenario assumes that a user of user terminal 2 purchases prepared foods, bento boxes, and other items that may be discounted based on the time elapsed since their manufacture at a store T that handles food products, such as supermarkets. The user is assumed to have completed member registration in a dedicated application program. Upon member registration, the user is issued a unique member ID. This member ID is set, for example, on user terminal 2, which is owned by the user. Note that user terminal 2 may be a terminal owned by the user or a terminal lent to the user by store T. The user may also purchase products through user terminal 2 without registering as a member.

[0054] The user checks in via a check-in device installed at the entrance of store T. For example, when the user launches a dedicated application program on user terminal 2, a barcode or two-dimensional code representing the member ID is displayed on display device 25. The user reads the member ID via the check-in device and checks in. The member ID may be recorded on RFID (Radio Frequency Identification), an IC card, or a magnetic card, etc. Check-in using the member ID may be performed by a publicly known process.

[0055] The user reads AR markers attached to products displayed in store T via camera 28 and displays AR content based on the AR markers on display device 25.

[0056] User terminal 2 processes the following information in response to the user reading the AR marker. In the following description, "in response" may be read as "based on".

[0057] Figure 7 is a flowchart illustrating the information processing procedure by the user terminal 2 according to the first embodiment.

[0058] The first acquisition unit 210 acquires information about the product based on an image related to the product captured via the camera 28 (ACT1). In ACT1, for example, the first acquisition unit 210 acquires the product code based on an image related to the product captured via the camera 28. The first acquisition unit 210 performs image processing on the product code image included in the captured image related to the product to acquire the product code of the product. The first acquisition unit 210 acquires the manufacturing date and time from the image related to the product based on the image related to the product captured via the camera 28. For example, the first acquisition unit 210 performs image processing on the manufacturing date and time image included in the captured image related to the product to acquire the manufacturing date and time of the product. The first acquisition unit 210 acquires AR marker image information from the image related to the product based on the image related to the product captured via the camera 28. For example, the first acquisition unit 210 acquires AR marker image information by image processing on the AR marker image included in the captured image related to the product. The first acquisition unit 210 acquires information about a temporary object associated with the acquired AR marker image information stored in the auxiliary storage device 23.

[0059] The display control unit 212 displays a temporary object on the display device 25 based on an image related to the product captured via the camera 28 (ACT2). In ACT2, for example, the display control unit 212 displays the temporary object on the display device 25 based on the temporary object information acquired by the first acquisition unit 210.

[0060] The second acquisition unit 211 acquires product incentive information based on the manufacturing date and time (ACT3). In ACT3, for example, the second acquisition unit 211 acquires product incentive information from server 1 based on the product code and the manufacturing date and time.

[0061] The display control unit 212 displays the main object containing incentive information on the display device 25 based on the incentive information acquired by the second acquisition unit 211 (ACT4). In ACT4, for example, the display control unit 212 changes the display mode of the main object according to the incentive information. For example, the display control unit 212 changes the display mode of the main object based on the content of the incentive indicated by the incentive information. The display control unit 212 displays the main object if the incentive information indicates that an incentive will be given. The display control unit 212 may display the main object indicating that there is no incentive if the incentive information indicates that no incentive will be given, or it may not display the main object. If the display control unit 212 cannot acquire incentive information, it maintains the display of a temporary object until incentive information is acquired.

[0062] In addition, the processor 21 may output audio indicating incentive information via the speaker 26, either in addition to displaying the object or in lieu of displaying the object.

[0063] The registration unit 213 registers an incentive corresponding to the incentive information for the product based on the user's selection operation of this object (ACT5). In ACT5, for example, the registration unit 213 registers the product code, incentive information, and amount associated with the object selected by the user as registered product data.

[0064] Subsequently, the processor 21 calculates a discounted price by applying incentive information to the registered product price based on the registered product data. Based on the user's accounting instructions, the processor 21 performs known accounting processing based on the discounted price.

[0065] The display control unit 212 may display the actual object on the display device 25 without displaying the temporary object on the display device 25. In this example, the processing of ACT2 is omitted.

[0066] The auxiliary storage device 23 of the user terminal 2 may store product data for at least one product. In this example, in ACT3, for example, the second acquisition unit 211 calculates the elapsed time from the manufacturing date and time to the current date and time based on the manufacturing date and time acquired by the first acquisition unit 210. The second acquisition unit 211 acquires incentive information associated with the calculated elapsed time for the product code acquired by the first acquisition unit 210 from the product data stored in the auxiliary storage device 23.

[0067] The auxiliary storage device 23 of the user terminal 2 may also store registered product data. In this example, in ACT5, for example, the registration unit 213 updates the registered product data based on the product code and incentive information in response to a selection operation by the user. The registration unit 213 adds a record containing the product code, incentive information, and amount to the registered product data. The registration unit 213 retrieves the amount of the product associated with the product code from the product master.

[0068] (effect) The information processing device according to this embodiment can acquire the manufacturing date and time of a product based on an image related to the product taken by the camera unit, acquire incentive information for the product based on the manufacturing date and time, and display an object containing the incentive information on the display unit. This allows the information processing device to provide customers with incentive information that changes over time by displaying the incentive information corresponding to the product's manufacturing date and time as an object. Therefore, the information processing device can provide flexible incentives, such as product discounts, based on the capture of images related to the product. Furthermore, the information processing device can provide incentive information to customers without requiring store staff to attach discount stickers. Therefore, the information processing device can improve the work efficiency of store staff.

[0069] The information processing device according to this embodiment can display a temporary object on the display unit based on an image related to a product, and display a real object on the display unit based on incentive information. The information processing device displays the temporary object until incentive information is obtained, and displays the real object in response to the acquisition of the incentive information. In this way, the information processing device can make the customer aware that incentive information has not been obtained by displaying the temporary object. Therefore, the information processing device can make the customer aware that the image related to the product is not in focus and prompt them to operate to focus on it.

[0070] The information processing device according to this embodiment can change the display mode of the object according to incentive information. This allows the information processing device to display different versions of the object depending on the incentive information. Therefore, customers can visually recognize the differences in incentive information. For example, the information processing device hides the object when there is no incentive information. Therefore, customers can recognize, based on the object, that the desired product is one to which no incentive applies, such as a freshly made product with a short time elapsed since its manufacture.

[0071] The information processing device according to this embodiment can register an incentive corresponding to incentive information to a product based on the user's selection operation of this object. As a result, the information processing device can register a product by the user's selection operation of this object without requiring any further product registration work. Therefore, the information processing device can omit additional product registration processes such as product code recognition.

[0072] [Second Embodiment] The second embodiment will be described below with reference to the drawings. Components similar to those in the first embodiment are denoted by the same reference numerals, and their descriptions are omitted. In the second embodiment, the differences from the first embodiment will be described primarily.

[0073] (Example configuration) Figure 8 is a block diagram illustrating an information processing system 100 according to the second embodiment. The information processing system 100 includes a server 1 and a user terminal 2, similar to the first embodiment.

[0074] Server 1 is an electronic device that collects and processes data. Electronic devices include computers. Server 1 communicates freely with user terminal 2 via a network. Server 1 receives various data from user terminal 2 and outputs various data to user terminal 2. Server 1 manages product information in stores such as supermarkets, for example. An example configuration of Server 1 will be described later. Server 1 is an example of an information processing device.

[0075] User terminal 2 is an electronic device capable of communicating with other electronic devices. User terminal 2 is, for example, a device used by a customer in a store. For example, user terminal 2 is a smartphone or a tablet device. The term "customer" may be replaced with "user" or "person." An example configuration of user terminal 2 will be described later.

[0076] This section describes an example configuration for Server 1. The hardware configuration of Server 1 is the same as in the first embodiment.

[0077] The auxiliary storage device 13 includes an object data storage area 130. The object data storage area 130 stores data for at least one dummy object. The data for the dummy object represents data for a dummy object associated with AR marker image information, as described in the first embodiment.

[0078] The auxiliary storage device 13 includes a product data storage area 131. The product data storage area 131 stores product data relating to at least one product, as described in the first embodiment.

[0079] The auxiliary storage device 13 includes a registered product data storage area 132. The registered product data storage area 132 stores registered product data as described in the first embodiment.

[0080] Note that the hardware configuration of Server 1 is not limited to the configuration described above. Server 1 may, as appropriate, omit or change the above-mentioned components and add new components.

[0081] The various components implemented in processor 11 will now be described.

[0082] The processor 11 implements a first acquisition unit 110, a second acquisition unit 111, a display control unit 112, and a registration unit 113. Each part implemented in the processor 11 can also be called a function. Each part implemented in the processor 11 can also be said to be implemented in the control unit which includes the processor 11 and the main memory 12.

[0083] The first acquisition unit 110 acquires a product code based on an image related to the product captured via the camera 28 of the user terminal 2. The image related to the product includes an AR marker image, a manufacturing date and time image, and a product code image, as described in the first embodiment. For example, the first acquisition unit 110 acquires a product code from the user terminal 2 via the network in response to an image related to the product being captured via the camera 28 of the user terminal 2. The user terminal 2 performs image processing on the product code image included in the captured image related to the product to acquire the product code of the product. The user terminal 2 outputs the product code to the server 1 via the network. The product code image may also include text information. In the following description, "in response" may be read as "based on". Alternatively, the user terminal 2 may send the product code image included in the captured image related to the product to the server, and the server 1 may acquire the product code of the product by performing image processing on the received product code image.

[0084] The first acquisition unit 110 acquires the manufacturing date and time based on an image related to the product captured via the camera 28 of the user terminal 2. For example, the first acquisition unit 110 acquires the manufacturing date and time from the user terminal 2 via the network in response to an image related to the product being captured via the camera 28 of the user terminal 2. The user terminal 2 performs image processing on the manufacturing date and time image included in the captured image related to the product and acquires the manufacturing date and time of the product. The user terminal 2 outputs the manufacturing date and time to the server 1 via the network.

[0085] The first acquisition unit 110 acquires AR marker image information based on images related to the product captured via the camera 28 of the user terminal 2. For example, the first acquisition unit 110 acquires AR marker image information from the user terminal 2 via the network in response to an image related to the product being captured via the camera 28 of the user terminal 2. The user terminal 2 acquires AR marker image information by image processing of the AR marker image contained in the captured image related to the product. The user terminal 2 outputs the AR marker image information to the server 1 via the network. The first acquisition unit 110 acquires the AR marker image information from the user terminal 2 and acquires information about a temporary object associated with the acquired AR marker image information stored in the auxiliary storage device 13. In the following description, "acquisition" may be read as "reception," and "output" may be read as "transmission."

[0086] The second acquisition unit 111 acquires incentive information for a product based on the manufacturing date and time. For example, the second acquisition unit 111 acquires incentive information for a product based on the product code and manufacturing date and time acquired by the first acquisition unit 110. The second acquisition unit 111 acquires incentive information for a product in response to the acquisition of the product code and manufacturing date and time from the user terminal 2. For example, the second acquisition unit 111 calculates the elapsed time from the manufacturing date and time to the current date and time based on the manufacturing date and time acquired by the first acquisition unit 110. The current date and time indicates the date and time when an image related to the product was taken by the user terminal 2 via the camera 28. The current date and time may also be the date and time when at least one of the product code and manufacturing date and time was acquired by the first acquisition unit 110. The second acquisition unit 111 acquires incentive information associated with the calculated elapsed time for the product code acquired by the first acquisition unit 110 from the product data stored in the product data storage area 131.

[0087] The display control unit 112 displays a temporary object on the display device 25 based on an image related to the product captured via the camera 28 of the user terminal 2. In response to the acquisition of a temporary object based on the capture of an image related to the product by the first acquisition unit 110, the display control unit 112 outputs the temporary object to the user terminal 2 via the network. By outputting the temporary object to the user terminal 2, the display control unit 112 displays the temporary object on the display device 25. The user terminal 2 displays the temporary object on the display device 25 based on the acquisition of the temporary object from the server 1 via the network.

[0088] The display control unit 112 displays the object on the user terminal's display device 25 based on the incentive information acquired by the second acquisition unit 111. The display control unit 112 displays the object on the display device 25 in response to the acquisition of incentive information by the second acquisition unit 111. The display control unit 112 displays the object on the display device 25 by outputting the object to the user terminal 2. The display control unit 112 changes the display of the temporary object to the display of the actual object in response to the acquisition of incentive information. The display control unit 112 may replace the temporary object with the actual object, or it may gradually transition from the temporary object to the actual object. The user terminal 2 displays the object on the display device 25 based on the acquisition of the object from the server 1 via the network.

[0089] The display control unit 112 changes the display mode of the object according to the incentive information. For example, as described in the first embodiment, the display control unit 112 changes the display mode of the object based on the content of the incentive indicated by the incentive information.

[0090] The registration unit 113 registers an incentive corresponding to the incentive information to the product based on the user's selection operation of the product. For example, the user may select one of the at least one product displayed on the display device 25. In response to the user's selection operation, the registration unit 113 obtains the product code and incentive information associated with the selected product from the user terminal 2 via the network, associated with the product registration instruction. The registration unit 113 may obtain the product code and incentive information from the user terminal 2 simultaneously, associated with the product registration instruction, or at different times. The registration unit 113 updates the registered product data based on the product registration instruction. The registration unit 113 adds a record containing the product code, incentive information, and amount to the registered product data. The registration unit 113 obtains the amount of the product associated with the product code from the product master.

[0091] This section describes an example configuration for user terminal 2. The hardware configuration of user terminal 2 is the same as in the first embodiment.

[0092] The hardware configuration of user terminal 2 is not limited to the configuration described above. User terminal 2 may, as appropriate, omit or change the above-described components and add new components.

[0093] (Example of operation) The procedure for processing by Server 1 will be described below. In the following explanation focusing on Server 1, you may substitute Server 1 for Processor 11. Similarly, in the explanation focusing on User Terminal 2, you may substitute User Terminal 2 for Processor 21. The processing procedure described below is merely an example, and each process may be modified as much as possible. Furthermore, depending on the embodiment, steps in the processing procedure described below may be omitted, replaced, or added as appropriate.

[0094] Here, similar to the first embodiment, we assume that a user of user terminal 2 purchases prepared foods, bento boxes, and other items that are discounted according to the time elapsed since their manufacture at a store T that handles food products, such as a supermarket. The user is assumed to have completed member registration in a dedicated application program. Upon member registration, the user is issued a unique member ID. The member ID is set, for example, on user terminal 2 owned by the user. Note that user terminal 2 may be a terminal owned by the user, or a terminal lent to the user by store T. Note that the user may also purchase products via user terminal 2 without registering as a member.

[0095] The user checks in via a check-in device installed at the entrance of store T. For example, when the user launches a dedicated application program on user terminal 2, a barcode or two-dimensional code representing the member ID is displayed on display device 25. The user reads the member ID via the check-in device and checks in. The member ID may be recorded on an RFID, IC card, or magnetic card, etc. Check-in using the member ID may be performed by a publicly known process.

[0096] The user reads AR markers attached to products displayed in store T via camera 28 and displays AR content based on the AR markers on display device 25.

[0097] Server 1 processes the following information in response to the user reading the AR marker. In the following description, "in response" may be read as "based on".

[0098] Figure 9 is a flowchart illustrating the information processing procedure performed by Server 1 according to the second embodiment.

[0099] The first acquisition unit 110 acquires information about the product based on an image related to the product captured via the camera 28 of the user terminal 2 (ACT 11). In ACT 11, for example, the first acquisition unit 110 acquires the product code based on an image related to the product captured via the camera 28. The first acquisition unit 110 acquires the product code of the product output from the user terminal 2 via the network. The first acquisition unit 110 acquires the manufacturing date and time based on an image related to the product captured via the camera 28 of the user terminal 2. For example, the first acquisition unit 110 acquires the manufacturing date and time of the product output from the user terminal 2 via the network. The first acquisition unit 110 acquires AR marker image information based on an image related to the product captured via the camera 28 of the user terminal 2. For example, the first acquisition unit 110 acquires AR marker image information output from the user terminal 2 via the network. The first acquisition unit 110 acquires information about a temporary object associated with the acquired AR marker image information stored in the auxiliary storage device 13.

[0100] The display control unit 112 displays a temporary object on the display device 25 based on an image related to the product captured via the camera 28 of the user terminal 2 (ACT12). In ACT12, for example, the display control unit 112 outputs the temporary object to the user terminal 2 via the network based on the temporary object information acquired by the first acquisition unit 110. By outputting the temporary object to the user terminal 2, the display control unit 112 displays the temporary object on the display device 25.

[0101] The second acquisition unit 111 acquires product incentive information based on the manufacturing date and time (ACT 13). In ACT 13, for example, the second acquisition unit 111 acquires product incentive information from the product data storage area 131 based on the product code and the manufacturing date and time.

[0102] The display control unit 112 displays the object containing incentive information on the display device 25 based on the incentive information acquired by the second acquisition unit 111 (ACT 14). In ACT 14, for example, the display control unit 112 outputs the object to the user terminal 2 via the network. The display control unit 112 displays the object on the display device 25 by outputting the object to the user terminal 2. The display control unit 112 changes the display mode of the object according to the incentive information. For example, the display control unit 112 changes the display mode of the object based on the content of the incentive indicated by the incentive information. The display control unit 112 displays the object if the incentive information indicates that an incentive will be given. The display control unit 112 may display an object indicating that there is no incentive if the incentive information indicates that no incentive will be given, or it may not display the object. If the display control unit 112 cannot acquire incentive information, it maintains the display of a temporary object until incentive information is acquired.

[0103] In addition to displaying the object, or instead of displaying the object, the processor 11 may output audio indicating incentive information to the user terminal 2. In this case, the user terminal 2 may output audio indicating incentive information via the speaker 26.

[0104] The registration unit 113 registers an incentive corresponding to the incentive information to the product based on the user's selection operation of this object (ACT 15). In ACT 15, for example, the registration unit 113 registers the product code, incentive information, and amount associated with the object selected by the user as registered product data.

[0105] Subsequently, the processor 11 calculates a discounted amount by applying incentive information to the registered product price based on the registered product data. Based on the user's accounting instructions, the processor 11 performs known accounting processing based on the discounted price.

[0106] In ACT11, the first acquisition unit 110 acquires information about products from the user terminal 2 via the network, but is not limited to this. The first acquisition unit 110 may also acquire images related to products from the user terminal 2 via the network in response to an image related to a product being captured via the camera 28 of the user terminal 2. In this example, the first acquisition unit 110 acquires the product code by image processing of the product code image included in the product-related image acquired from the user terminal 2. The first acquisition unit 110 acquires the manufacturing date and time by image processing of the manufacturing date and time image included in the product-related image acquired from the user terminal 2. The first acquisition unit 110 acquires AR marker image information by image processing of the AR marker image included in the product-related image acquired from the user terminal 2.

[0107] (effect) The information processing device according to this embodiment can acquire the manufacturing date and time of a product based on an image related to the product taken by the camera unit, acquire incentive information for the product based on the manufacturing date and time, and display an object containing the incentive information on the display unit. This allows the information processing device to provide customers with incentive information that changes over time by displaying incentive information corresponding to the product's manufacturing date and time as an object. Therefore, the information processing device can provide flexible incentives such as product discounts based on the image taken of the product. Furthermore, the information processing device can provide incentive information to customers without requiring store staff to attach discount stickers. Therefore, the information processing device can improve the efficiency of store staff. Moreover, even if the user terminal becomes unusable due to battery depletion or other reasons, the product registration information is stored in the information processing device's auxiliary storage device, such as a server. Therefore, when the user terminal is used again, it can resume from the state in which product registration was performed before the battery ran out. Additionally, since the user terminal only needs to output the read information to the server and receive and display the information sent from the server, the processing load or memory capacity load on the user terminal can be reduced.

[0108] The information processing device according to this embodiment can display a temporary object on the display unit based on an image related to a product, and display a real object on the display unit based on incentive information. The information processing device displays the temporary object until incentive information is obtained, and displays the real object in response to the acquisition of the incentive information. In this way, the information processing device can make the customer aware that incentive information has not been obtained by displaying the temporary object. Therefore, the information processing device can make the customer aware that the image related to the product is not in focus and prompt them to operate to focus on it.

[0109] The information processing device according to this embodiment can change the display mode of the object according to incentive information. This allows the information processing device to display different versions of the object depending on the incentive information. Therefore, customers can visually recognize the differences in incentive information. For example, the information processing device hides the object when there is no incentive information. Therefore, customers can recognize, based on the object, that the desired product is one to which no incentive applies, such as a freshly made product with a short time elapsed since its manufacture.

[0110] The information processing device according to this embodiment can register an incentive corresponding to incentive information to a product based on the user's selection operation of this object. As a result, the information processing device can register products solely through the selection operation of this object, without requiring any further product registration work by the user. Therefore, the information processing device can omit additional product registration processes such as product code recognition.

[0111] [Other embodiments] In the first and second embodiments, AR markers were used to display the temporary object and the actual object, but the invention is not limited to this. Instead of AR markers, codes that can display the temporary object and the actual object, such as QR codes, barcodes, Chameleon Codes, etc., may be used.

[0112] Furthermore, incentive information is not limited to discount rate information. Incentive information may also include discount amount information, price information, points awarded information, or electronic money chargeback information.

[0113] While we have referred to objects that do not contain incentive information as provisional objects and objects that contain incentive information as actual objects, we are not limited to this. Either an object that does not contain incentive information or an object that contains incentive information may be referred to as the first object and the other as the second object.

[0114] The information processing device may be implemented as a single device, such as server 1 or user terminal 2, or as multiple devices with distributed functions.

[0115] The program may be transferred while stored on an electronic device, or it may be transferred without being stored on an electronic device. In the latter case, the program may be transferred via a network, or it may be transferred while recorded on a recording medium. The recording medium is a non-temporary, tangible medium. The recording medium is a computer-readable medium. The recording medium can be any medium that is capable of storing a program and is readable by a computer, such as a CD-ROM or memory card, and its form is not limited.

[0116] In addition, several embodiments of the present invention have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0117] 1...Server, 2...User terminal, 11...Processor, 12...Main memory, 13...Auxiliary storage device, 14...Communication interface, 21...Processor, 22...Main memory, 23...Auxiliary storage device, 24...Communication interface, 25...Display device, 26...Speaker, 27...Input device, 28...Camera, 100...Information processing system, 110...First acquisition unit, 111...Second acquisition unit, 112...Display control unit, 113...Registration unit, 130...O Object data storage area, 131... Product data storage area, 132... Registered product data storage area, 210... First acquisition unit, 211... Second acquisition unit, 212... Display control unit, 213... Registration unit, 230... Object data storage area, A... AR image, Aa~Ab... AR images, AA... AR marker image, AB... Manufacturing date and time image, AC... Product code image, Ba~Bb... Temporary object, Ca~Cb... Real object, Pa~Pb... Product, Sc... Screen.

Claims

1. A first acquisition unit acquires the manufacturing date and time of the product based on an AR marker image related to the product captured by the photography unit, A second acquisition unit that acquires incentive information for the product based on the manufacturing date and time, A display control unit that displays the object, which is AR content including the aforementioned incentive information, on the display unit, Equipped with, The display control unit displays a temporary object, which is AR content, on the display unit based on the AR marker image, and changes the display of the temporary object to the display of the actual object based on the fact that the incentive information has been acquired by the second acquisition unit. Information processing device.

2. The information processing apparatus according to claim 1, wherein the display control unit changes the display mode of the object according to the incentive information.

3. The information processing apparatus according to claim 1, further comprising a registration unit that registers an incentive corresponding to the incentive information to the product based on a user's selection operation of the object.

4. A method executed by an information processing device, Based on the AR marker image related to the product captured by the photography unit, the manufacturing date and time of the said product are obtained. To obtain incentive information for the said product based on the said manufacturing date and time, The object, which is AR content containing the aforementioned incentive information, is displayed on the display unit. Based on the AR marker image, a temporary object which is AR content is displayed on the display unit, and based on the acquisition of the incentive information, the display of the temporary object is changed to the display of the actual object. An information processing method comprising the following:

5. On the computer, A first acquisition function that acquires the manufacturing date and time of the product based on an AR marker image related to the product captured by the photography unit, A second acquisition function that acquires incentive information for the product based on the manufacturing date and time, A display control function that displays the object, which is AR content including the aforementioned incentive information, on the display unit, An information processing program for executing, The display control function displays a temporary object, which is AR content, on the display unit based on the AR marker image, and changes the display of the temporary object to the display of the actual object based on the acquisition of the incentive information by the second acquisition function. Information processing program.

Images