Information processing device, information processing method, and program

Abstract

To provide useful information to customers in a store.SOLUTION: An information processing device disclosed herein includes an identification unit, an acquisition unit, and an information provision unit. The identification unit identifies at least one product displayed in a store based on images captured by a camera carried by a user moving through the store. The acquisition unit acquires product information for the identified product. If product information for multiple products is acquired, the information provision unit outputs comparison information about comparison of the product information for the multiple products.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A technology related to this disclosure is disclosed in Patent Document 1. The technology disclosed in Patent Document 1 uses a light-transmitting display of smart glasses to display various information as augmented reality in the user's field of vision. For example, the technology disclosed in Patent Document 1 displays an arrow image indicating the route from the user's current location to the location of a specific food ingredient displayed in a store. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2023 / 112519 Summary of the Invention [Problem to be solved by the invention]

[0004] Providing useful information to customers in a store is expected to have effects such as increased sales. Previous technologies have also provided various types of information to customers in a store. However, there is still room for improvement. One example of the purpose of this disclosure is to provide a technology for providing useful information to customers in a store. [Means for solving the problem]

[0005] According to this disclosure: an identification means for identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; an acquisition means for acquiring product information of the identified product; an information providing means for outputting comparison information for comparing the product information of the plurality of products when the product information of the plurality of products is acquired; An information processing device having the above configuration is provided.

[0006] Also, according to this disclosure: One or more computers Identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; Obtain product information for the identified product; There is provided an information processing method for outputting, when the product information of a plurality of products is acquired, comparison information for comparing the product information of the plurality of products.

[0007] Also, according to this disclosure: Computer, an identification means for identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; an acquisition means for acquiring product information of the identified product; an information providing means for outputting, when the product information of a plurality of products is acquired, comparison information for comparing the product information of the plurality of products; A program is provided to function as a [Effects of the Invention]

[0008] According to one aspect of the present disclosure, a technology is realized that provides useful information to customers in a store. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional block diagram of an information processing apparatus. [Figure 2] 10 is a flowchart illustrating an example of a processing flow of an information processing device. [Figure 3] FIG. 1 illustrates an example of a hardware configuration of an information processing device. [Figure 4] FIG. 10 is a diagram illustrating an example of an image acquired by the information processing apparatus. [Figure 5] FIG. 10 is a diagram illustrating an example of information output by an information processing device. [Figure 6] FIG. 10 is a diagram illustrating another example of information output by the information processing device. [Figure 7] FIG. 10 is a diagram illustrating another example of information output by the information processing device. [Figure 8] FIG. 10 is a diagram illustrating another example of information output by the information processing device. [Figure 9] FIG. 10 is a diagram illustrating another example of information output by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0011] <<First embodiment>> Fig. 1 is a functional block diagram showing an overview of an information processing device 10. Fig. 2 is a flowchart showing an example of the flow of processing executed by the information processing device 10.

[0012] 1, the information processing device 10 includes an identifying unit 11, an acquiring unit 12, and an information providing unit 13. These functional units execute the processing of the flowchart in FIG.

[0013] In S10, the identification unit 11 identifies at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store. In S11, the acquisition unit 12 acquires product information of the product identified in S11. In S12, when the product information of a plurality of products is acquired, the information providing unit 13 outputs comparison information for comparing the product information of the plurality of products.

[0014] When similar products are sold, it can be difficult to understand the differences between them. By using the information processing device 10 that outputs comparison information that compares product information for multiple products, customers can easily understand the differences between them and the characteristics of each product.

[0015] <<Second embodiment>> <Summary> The processing device 10 of the second embodiment is a specific implementation of the configuration of the information processing device 10 of the first embodiment, which will be described in detail below.

[0016] <Hardware configuration> First, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-loaded when the device is shipped, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0017] FIG. 3 is a block diagram illustrating an example of the hardware configuration of an information processing device 10. As shown in FIG. 3, the information processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The information processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the information processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0018] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, and touch panel. Examples of output devices include a display, projector, speaker, printer, and mailer. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0019] <Functional configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 1 shows an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an identification unit 11, an acquisition unit 12, and an information provision unit 13.

[0020] The "information processing device 10" is a user terminal owned by a user.

[0021] A "user" is a customer who comes to shop at a store. The user carries a user terminal and moves around the store. The user terminal may be owned by the user, or may be owned by the store and loaned to the user by the store. By installing a predetermined program in the user terminal, the identification unit 11, acquisition unit 12, and information provision unit 13 are realized within the user terminal.

[0022] The "user terminal" has at least a camera function, an input / output function, a communication function, and a calculation function. The user terminal may be an AR (Augmented Reality) terminal having an AR function.

[0023] The user terminal may be a wearable terminal. For example, the user terminal may be smart glasses, a head-up display, a head-mounted display, or the like. In this case, the camera function of the user terminal is installed in the user terminal at a position and orientation that captures the user's field of view when worn by the user. The input / output function of the user terminal displays various information superimposed on the user's field of view (real world). The wearable terminal may be equipped with bone conduction earphones in the earpieces. Various audio guidance from the store may be output via these earphones.

[0024] Alternatively, the user terminal may be a handheld terminal such as a smartphone, tablet terminal, or mobile phone. In this case, the input / output function of the user terminal displays an image generated by the camera function on the display in real time. At this time, the display functions as a camera viewfinder. The input / output function of the user terminal then displays various information superimposed on the image of the real world displayed on the display (viewfinder).

[0025] The identification unit 11 identifies at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store.

[0026] The "camera carried by the user" refers to the camera mounted on the user terminal described above. The camera may continuously generate multiple images while the user is moving around the store. Alternatively, the camera may continuously generate multiple images while the user has the camera function turned on and is taking pictures.

[0027] FIG. 4 shows an example of an image I generated by a camera carried by a user. FIG. 4 shows a product P displayed on a display shelf N. On the display shelf N, text information R and a code Q for each product P are displayed, linked to the display area of ​​each product P. The text information R includes the product name, price, etc. The code Q indicates various information such as product identification information. The text information R and the code Q may be displayed, for example, on a shelf label, or may be displayed by other methods.

[0028] There are various types of "products." Products may be foods, beverages, daily necessities, electrical appliances, or other products.

[0029] Next, the "process of identifying at least one of the products displayed in the store" will be described. In this process, the identification unit 11 identifies the product shown in the image or the product whose related information is shown in the image. The user directs their gaze toward the product or related information in which they are interested and visually recognizes it. Therefore, when the user terminal is a wearable terminal and captures the user's field of view, the product or related information in which the user is interested tends to be included in the image. Furthermore, when the user terminal is a handheld terminal such as a smartphone, the user points the camera at the product or related information in which they are interested and captures the image.

[0030] Therefore, the identifying unit 11 that identifies the product that appears in the image or the product whose related information appears in the image can identify the product in which the user is interested.

[0031] There are various means for identifying products shown in an image, and any widely known technology can be employed. For example, the appearance feature quantities of each of a plurality of products displayed in a store may be stored in advance in a predetermined storage device. The predetermined storage device may be provided within the information processing device 10, or may be provided in an external device configured to be able to communicate with the information processing device 10 (the same applies hereinafter). Then, the identification unit 11 may identify the products shown in the image based on the results of comparing the appearance feature quantities of the objects included in the image with the appearance feature quantities of the products registered in advance.

[0032] The product-related information may be text information of product identification information (product name, product code, etc.) or a code (barcode, two-dimensional code, etc.) indicating the product identification information. For example, such related information may be displayed on product display shelves. In addition, such related information may be displayed on bulletin boards, walls, hanging advertisements, etc. in stores.

[0033] In this case, the identification unit 11 can analyze text information (related information) in the image using, for example, OCR (Optical Character Recognition) technology and identify the product identification information. Additionally, the identification unit 11 can decode codes (related information) in the image to identify the product identification information.

[0034] The identification unit 11 may identify all products that appear in the image even for a moment (or products that appear with related information). However, in this case, the number of identified products may become enormous. Therefore, the identification unit 11 may identify products that appear in the image continuously for a predetermined period of time or more (or products that appear with related information in the image continuously for a predetermined period of time or more).

[0035] The acquisition unit 12 acquires product information of the product identified by the identification unit 11. Product information for each of a plurality of products is registered in advance in a server. The acquisition unit 12 transmits a request for product information of the product identified by the identification unit 11 to the server. The acquisition unit 12 then acquires the product information returned from the server. The server may be a server managed by each store, a server managed by the store headquarters, or another server.

[0036] "Product information" includes various information related to products. Product information includes at least one of information provided by the manufacturer, information set by the store headquarters, information set by each store, and information obtained from an external server. Product information can include at least one of the following types of information, for example:

[0037] ·Product name Manufacturer name ·price Manufacturer's suggested price -Various nutrient contents Calories (e.g. manufacturer specifications) Calories per 100g - Ratio of the five basic tastes (salty, sweet, sour, bitter, umami) -Content of various ingredients Product specifications Product popularity rankings (popularity rankings at each store, popularity rankings across all convenience stores, etc.) ·POP(Point of purchase advertising)

[0038] When product information of a plurality of products is acquired, the information providing unit 13 outputs comparison information that compares the product information of the plurality of products. The information providing unit 13 can generate and output a screen that lists the product information of the plurality of products as the comparison information. Note that the information providing unit 13 can use AR technology to display various information such as the comparison information superimposed on the real world.

[0039] Fig. 5 shows an example of comparison information output by the information providing unit 13. The information within the dotted frame M is information displayed superimposed on the real world. Information other than the information within the dotted frame M is real-world information that the user sees directly through a wearable device such as smart glasses, or real-world information displayed on a display (finder) of a smartphone or the like. The display method using the dotted frame M is the same in Figs. 6 to 9.

[0040] In the example of FIG. 5, product names and prices are displayed as comparison information. Although not shown, the comparison information may include other product information. For example, the comparison information may include a radar chart showing the balance of the ratios of the five basic flavors. In addition, the comparison information may include a product popularity ranking. In addition, the comparison information may include calories, calories per 100g, the content of various nutrients, etc. In addition, the comparison information may include a radar chart showing the balance of the three major nutrients (carbohydrates, protein, and fat). Furthermore, the comparison information may indicate the content of nutrients other than the three major nutrients using colored text, etc.

[0041] It should be noted that, when product information for a single product has been acquired but product information for multiple products has not been acquired, the information providing unit 13 can output the product information for that product.

[0042] Fig. 6 shows an example of information output by the information providing unit 13. In the example of Fig. 6, a POP (product information) of a product is displayed superimposed on the real world. Also, in the example of Fig. 6, inventory information of the product is displayed. A method for generating this inventory information will be described in the following embodiment.

[0043] The information providing unit 13 can display other information not shown. For example, the information providing unit 13 identifies the user's age and gender and determines the user's basal metabolic rate from the identification results. The information providing unit 13 then calculates the target intake amounts of various nutrients for each user based on the determination results. The information providing unit 13 can then display warning information indicating that a product exceeds the target intake amount or exceeds the target intake amount by more than a threshold value. For example, the information providing unit 13 can display the "name of the nutrient that exceeds the target intake amount or exceeds the target intake amount by more than a threshold value," the "target intake amount of the nutrient," the "content of the nutrient in the product," etc.

[0044] There are various ways to identify the user's age and gender, such as by receiving input from the user. The process of determining the user's basal metabolic rate from the user's age and gender, and the process of calculating the target intake amounts of various nutrients based on the basal metabolic rate, are realized based on calculation algorithms stored in advance in a predetermined storage device.

[0045] Next, we will explain the case where "product information for multiple products is acquired." When product information for multiple products is acquired, this is literally the case where product information for multiple products is acquired by the information processing device 10. As a variant, when product information for multiple products is acquired, it may be defined as a case where any one of the following conditions 1 to 4 is satisfied.

[0046] (Condition 1) Product information for multiple products was obtained from one image (Condition 2) Product information for a product is acquired within a specified time after product information for a product is acquired. (Condition 3) Product information for multiple products was obtained while the user was in the same sales area. (Condition 4) After product information for a certain product is acquired, the user's travel distance is within a threshold and the next product information is acquired.

[0047] First, we will explain condition 1. A user simultaneously views or photographs multiple products and related information that they wish to compare. As a result, as shown in Figure 4, a single image generated by the user terminal simultaneously contains all of the multiple products and related information. Product information for multiple products is then obtained from a single image.

[0048] Therefore, when product information of multiple products is acquired from one image, the user may want to compare the product information of the multiple products. Therefore, the information providing unit 13 can output comparison information when condition 1 is satisfied.

[0049] Next, condition 2 will be described. A user may sequentially view or photograph multiple products and their related information that they wish to compare, and compare them visually. For example, immediately after viewing a first product and its related information, the user views a second product and its related information that they wish to compare with the first product. Typically, the time between viewing the first product and its related information and viewing the second product and its related information is relatively short.

[0050] Therefore, if product information for a certain product is acquired and then product information for a next product is acquired within a predetermined time, the user may want to compare the product information for the multiple products. Therefore, the information providing unit 13 can output comparison information when condition 2 is satisfied.

[0051] Next, condition 3 will be described. A user may sequentially view or photograph multiple products and their related information sold in the same sales area to compare them. For example, after viewing a first product and its related information in a certain sales area, the user may view a second product and its related information in the same sales area that the user wants to compare with the first product.

[0052] Therefore, when product information of multiple products is acquired while the user is in the same sales area, the user may want to compare the product information of the multiple products. Therefore, the information providing unit 13 can output comparison information when condition 3 is satisfied.

[0053] When Condition 3 is adopted, the information providing unit 13 identifies the user's current location. Then, the information providing unit 13 determines whether the user continues to be in the same sales area based on the identified user's current location and a floor map showing each of the store's multiple sales areas, which is stored in advance in a predetermined storage device. Identifying the user's current location can be achieved using any well-known technology. For example, the information providing unit 13 may identify the user's current location using technology such as a global positioning system (GPS) or a beacon. Alternatively, the information providing unit 13 may identify the user's current location by detecting a landmark from an image generated by the user terminal. A landmark is a location marker that is stored in advance in a predetermined storage device. Alternatively, the information providing unit 13 may identify the user's current location by acquiring an image of the store interior and detecting the user from the image. In this case, the user's appearance information is stored in advance in a predetermined storage device.

[0054] Next, condition 4 will be described. Generally, multiple products that can be compared with each other are displayed in close proximity to each other. A user may sequentially view or photograph multiple products and their related information that are displayed in close proximity to each other and compare them. For example, after viewing a first product and its related information, a user may view a second product and its related information that are displayed nearby in order to compare them with the first product.

[0055] Therefore, if the user acquires product information for a certain product while the user's movement distance is within a threshold after acquiring product information for the next product, the user may want to compare the product information for the multiple products. Therefore, the information providing unit 13 can output comparison information when condition 4 is satisfied.

[0056] When Condition 4 is adopted, the information providing unit 13 identifies the current location of the user. Then, the information providing unit 13 calculates the user's travel distance based on the identified current location of the user. Identification of the user's current location can be realized using any widely known technology. For example, the above-mentioned technology may be adopted.

[0057] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of FIG.

[0058] First, a user carries a user terminal (information processing device 10) and moves around the store. The user terminal may continuously generate multiple images while the user is moving around the store. Alternatively, the user terminal may continuously generate multiple images while the user turns on the camera function and takes pictures.

[0059] The information processing device 10 identifies at least one of the products displayed in the store based on an image generated by a camera (a camera mounted on a user terminal) carried by a user moving around the store (S10). For example, the information processing device 10 identifies the product shown in the image or the product whose related information is shown in the image.

[0060] Next, the information processing device 10 acquires product information of the product identified in S10 (S11). For example, the information processing device 10 transmits a request for product information of the product identified in S10 to a predetermined server. Then, the information processing device 10 acquires the product information returned from the server.

[0061] Next, the information processing device 10 outputs the acquired product information. When product information for a plurality of products is acquired, the information processing device 10 outputs comparison information that compares the product information for the plurality of products (S12). Note that when product information for a plurality of products is not acquired but product information for one product is acquired, the information processing device 10 can output the product information for that product. The information processing device 10 can display information superimposed on the real world using AR technology.

[0062] "Action and effect" According to the information processing device 10 of the second embodiment, the same effects as those of the information processing device 10 of the first embodiment are achieved.

[0063] Furthermore, the information processing device 10 of the second embodiment can output a screen in which product information for a plurality of products is arranged as comparison information. With such an information processing device 10, the user can easily compare a plurality of products.

[0064] Furthermore, the information processing device 10 of the second embodiment can output the comparison information when any one of the above-described conditions 1 to 4 is satisfied. According to such an information processing device 10, the comparison information can be output at an appropriate timing desired by the user.

[0065] <<Third embodiment>> The information processing device 10 of the third embodiment uses AR technology to display information superimposed on the real world, and changes the display mode of the displayed information based on at least one of the type and amount of information to be displayed. This will be described in detail below.

[0066] The information providing unit 13 uses AR technology to display information superimposed on the real world. The displayed information is the comparison information and product information described in the second embodiment. The information providing unit 13 then changes the display mode of the displayed information based on at least one of the type and amount of information to be displayed.

[0067] For example, the information providing unit 13 can change the transparency (display mode) of the information to be displayed based on at least one of the type and amount of information to be displayed.

[0068] The information providing unit 13 can set a higher transparency when displaying the first type of information than when displaying the second type of information. The first type of information and the second type of information are defined in advance and stored in a predetermined storage device. For example, information of relatively high importance may be defined as the second type, and information of relatively low importance may be defined as the first type. Alternatively, the information providing unit 13 may define information with a relatively small amount of information (e.g., number of characters) as the second type, and information with a relatively large amount of information as the first type.

[0069] Other configurations of the information processing device 10 of the third embodiment are similar to those of the information processing device 10 of the first and second embodiments.

[0070] According to the information processing device 10 of the third embodiment, the same effects as those of the information processing device 10 of the first and second embodiments are realized.

[0071] Furthermore, according to the information processing device 10 of the third embodiment, the display mode of the information to be displayed can be changed based on at least one of the type and amount of information to be displayed. For example, the information processing device 10 can increase the transparency of information with relatively low importance compared to information with relatively high importance. In this way, it is possible to prevent the inconvenience of information with relatively low importance obstructing the view of the real world. On the other hand, the transparency of information with relatively high importance can be reduced, thereby making its presence more prominent to the user.

[0072] Furthermore, the information processing device 10 can increase the transparency of information with a relatively large amount of information compared to information with a relatively small amount of information, thereby preventing the inconvenience of information with a large amount of information obstructing much of the view in the real world.

[0073] <<Fourth embodiment>> The information processing device 10 of the fourth embodiment has a function of informing the user of the display position of a product selected by the user, which will be described in detail below.

[0074] The identification unit 11 Processing to identify the product selected by the user -Process to identify the user's current location within the store -Process to identify the route from the user's current location to the display location of the product selected by the user Each step of the process will be explained below.

[0075] "Process to identify the product selected by the user" For example, the user selects a product that the user is interested in. The information processing device 10 can use any well-known technology to accept the product selection from the user.

[0076] In one example, users who wish to use the display position guidance service register in advance, and through this user registration, user identification information for identifying multiple users from each other is assigned to each user.

[0077] At any time after user registration, the user logs in to the server of the display position guidance service using their own user identification information. This login can be performed from a predetermined application or a predetermined web page. After logging in, the user then selects and registers at least one product. Through this process, the product selected by the user is registered in the server of the display position guidance service in association with the user identification information.

[0078] Then, a user who visits a store inputs their own user identification information into the user terminal (information processing device 10) they carry while moving around the store and logs in. The identification unit 11 requests information indicating products registered in association with the input user identification information from the server of the display position guidance service. The identification unit 11 then identifies the product selected by the user based on the information returned from the server of the display position guidance service.

[0079] Note that biometric information for authentication (face information, fingerprint information, voiceprint information, iris information, etc.) may be registered during the user registration. Then, the above-mentioned login may be realized by biometric authentication.

[0080] As another example, the identification unit 11 may receive an input from the user to select at least one product via an input device included in the information processing device 10. The identification unit 11 identifies the product selected by the input. Examples of the input device include, but are not limited to, a touch panel, a microphone, and a physical button. In this example, the user can use the display position guidance service without prior user registration.

[0081] "Process to identify the user's current location within the store" The identification unit 11 can identify the current location of the user using any well-known technology. For example, the identification unit 11 may identify the current location of the user using technology such as GPS or beacons. Alternatively, the identification unit 11 may identify the current location of the user by detecting landmarks from an image generated by the user terminal. Landmarks are location markers that are stored in advance in a predetermined storage device. Alternatively, the identification unit 11 may identify the current location of the user by acquiring an image of the inside of a store and detecting the user from the image. In this case, the user's appearance information is stored in advance in a predetermined storage device.

[0082] "Process to identify the route from the user's current location to the display location of the product selected by the user" In one example, the identification unit 11 can calculate a movement route based on the current location of the user and the display position of the product selected by the user. In another example, an external device different from the information processing device 10 calculates a movement route based on the current location of the user and the display position of the product selected by the user. Then, the identification unit 11 can acquire the movement route calculated from the external device.

[0083] The display position of the product selected by the user is specified based on "display information indicating the display position of each of a plurality of products in the store" that is stored in advance in a predetermined storage device.

[0084] The travel route is calculated based on the user's current location (starting point), the display location of the product selected by the user (destination point), and floor map information. Calculation of the travel route using such information can be achieved using any widely known technology. Travel route calculation algorithms that calculate a travel route from a specified starting point to a destination point are widely used. Using these technologies, it is possible to calculate a travel route from the user's current location to the display location of the product selected by the user.

[0085] The information providing unit 13 outputs information indicating the travel route identified by the identifying unit 11. For example, the information providing unit 13 may output an image showing the travel route with an arrow or the like on a floor map.

[0086] As a modification of the fourth embodiment, the information providing unit 13 may display information indicating the types of products displayed on each display shelf N, superimposed on the real world, as shown in FIG.

[0087] When realizing a display such as the example of FIG. 7 , the identification unit 11 has a function of identifying the display shelves N included in the image. Identification of the display shelves N included in the image can be realized using any technology. For example, identification information (e.g., a code, a serial number, etc.) may be attached to each display shelf N. The identification unit 11 may then identify the display shelves N in the image by detecting the identification information from the image. The identification unit 11 may also narrow down the display shelves N appearing in the image by using the user's current location. Alternatively, the identification unit 11 may identify the display shelves N in the image by identifying at least one product displayed on each display shelf N. In this case, the identification unit 11 identifies at least one product displayed on each display shelf N, and then identifies the display shelf N linked to the identified product based on linking information linking each display shelf N with the product displayed on each display shelf N.

[0088] After identifying the display shelf N in the image, the identification unit 11 identifies the product type of the product displayed on the identified display shelf N based on information indicating the product type of the product displayed on each display shelf N that is stored in advance in a predetermined storage device.

[0089] As a modification of the display shown in Fig. 7, the information providing unit 13 may display information indicating the type of product displayed on each display shelf N, linked only to the display shelf N that displays the product selected by the user. In other words, the information providing unit 13 does not need to display information indicating the type of product displayed on a display shelf N that does not display the product selected by the user, as shown in Fig. 7. In this way, it is possible to present to the user only the information that the user truly needs.

[0090] Furthermore, only when the product selected by the user is in stock, the information providing unit 13 may output information indicating the travel route identified by the identifying unit 11 or information indicating the type of product on display as shown in Fig. 7. When the product selected by the user is out of stock, the information providing unit 13 may not output information indicating the travel route identified by the identifying unit 11 or information indicating the type of product on display as shown in Fig. 7, but may instead output information in text or the like indicating that the product is out of stock.

[0091] "Out of stock" means "out of stock on the display shelf" or "out of stock on both the display shelf and the back yard." A method for detecting such an "out of stock" situation will be described in the fifth embodiment.

[0092] The information processing device 10 may also analyze images generated by a user terminal or images displayed in a store to identify the product picked up by the user. This identification can be achieved using any well-known technology. The information providing unit 13 may then output warning information if the "product picked up by the user" is different from the "product selected by the user." For example, the information providing unit 13 may output text such as "You are picking up a product different from the product you previously selected." The information providing unit 13 may output the above-described warning information when the product selected by the user is located within a predetermined distance from the user's current location. That is, the information providing unit 13 may output the above-described warning information when the user picks up a different product from the product selected by the user, even though the product selected by the user is located within a predetermined distance from the user's current location.

[0093] Other configurations of the information processing device 10 of the fourth embodiment are similar to those of the information processing device 10 of the first to third embodiments.

[0094] According to the information processing device 10 of the fourth embodiment, the same effects as those of the information processing devices 10 of the first to third embodiments are realized.

[0095] Furthermore, the information processing device 10 of the fourth embodiment can guide the user to the display location of the product selected by the user, thereby preventing the user from purchasing the product because they do not know where the product is displayed.

[0096] <<Fifth embodiment>> The information processing device 10 of the fifth embodiment has a function of outputting information indicating whether or not the product is in stock in the backyard, which will be described in detail below.

[0097] The identifying unit 11 identifies whether or not each product is in stock in the backyard.

[0098] The "back of house" is a different location from the "sales floor" of a store. Products are displayed on shelves in the sales floor, but products that cannot be displayed on the shelves may be stored in the back of house. Generally, customers are not allowed to enter the back of house. For this reason, customers are unable to grasp the inventory status of products in the back of house.

[0099] There are various ways to identify whether each product is in stock in the backyard. An example will be described below, but the present invention is not limited to this.

[0100] For example, backyard inventory information indicating the inventory status in the backyard may be generated in advance and stored in a predetermined storage device. For example, a store clerk may grasp the inventory quantity of each product in the backyard and register it as backyard inventory information. Then, the identifying unit 11 may identify whether each product is in stock in the backyard based on the backyard inventory information.

[0101] Alternatively, a camera may be installed in the backyard to capture images of the product storage area in the backyard. The identifying unit 11 may then identify whether each product is in stock in the backyard by detecting the product in the image captured of the backyard. The identifying unit 11 can detect the product in the image captured of the backyard using product detection technology.

[0102] Alternatively, if each product has an RFID tag attached, a reader / writer that reads the tag information may be installed in the backyard. The tag information includes product identification information for each product. The identifying unit 11 may then identify whether each product is in stock in the backyard based on the results of reading by the reader / writer.

[0103] The information providing unit 13 outputs information indicating whether or not each product is in stock in the back yard. Fig. 6 shows an example of the information output by the information providing unit 13. "In stock" in Fig. 6 indicates, as an example, whether or not there is stock in the back yard.

[0104] Note that "In Stock" in Figure 6 may indicate whether or not there is stock on the display shelf and in the back yard. That is, if there is stock on at least one of the display shelf and the back yard, the display may read "In Stock." If there is no stock on either the display shelf or the back yard, the display may read "Out of Stock."

[0105] Here, a modification of the fifth embodiment will be described.

[0106] The identification unit 11 may identify off-display items, which are items that are out of stock on the display shelves, based on an image taken of the display shelves. The identification unit 11 may then identify whether the off-display items are in stock in the back yard. In this case, the identification unit 11 does not need to identify whether items that are not off-display items, i.e., "items in stock on the display shelves," are in stock in the back yard. There is little need to check whether items that are in stock on the display shelves are in stock in the back yard. By executing a process to check whether only necessary items are in stock in the back yard, the processing burden on the computer can be reduced.

[0107] Identifying products that are no longer on display can be achieved based on images of the display shelves. The identification unit 11 can identify products that are no longer in stock on the display shelves by identifying products that can no longer be detected in images of the display shelves. "No longer able to be detected in images of the display shelves" means that a product was previously detectable in the images, but has now become undetectable.

[0108] In this modified example, the information providing unit 13 outputs information indicating whether or not there is stock of an undisplayed product in the back yard. The information providing unit 13 does not need to output information indicating whether or not there is stock of a product that is not an undisplayed product, i.e., a "product in stock on a display shelf," in the back yard.

[0109] Next, an example of information output in this modified example will be shown.

[0110] The information providing unit 13 can output information about undisplayed products using AR technology. For example, the information providing unit 13 may display information about undisplayed products in the display area of ​​the undisplayed product on display shelf N (the area in display shelf N allocated to the undisplayed product), as shown in Fig. 8 .

[0111] In the example of Figure 8, an image of an undisplayed product is displayed in a display area for undisplayed products. This image allows the user to easily identify the product (undisplayed product) that was being sold in the display area where no product was displayed.

[0112] 8, information indicating whether or not the item that is not on display is in stock in the back yard ("In stock: Please ask a store clerk" in the figure) is displayed. This information allows the user to easily understand whether or not the item that is out of stock on display shelf N is in stock in the back yard.

[0113] The information providing unit 13 may display information about off-display items only for off-display items that are in stock in the back yard, as shown in FIG. 8. The information providing unit 13 may not need to display information about off-display items as shown in FIG. 8 for off-display items that are not in stock in the back yard. From the perspective of preventing the inconvenience of losing sales opportunities because stock is available in the back yard but the shelves are not replenished late, it is sufficient to display information about off-display items only for off-display items that are in stock in the back yard, as shown in FIG. 8. By suppressing the display of unnecessary information, the processing load on the computer can be reduced. Furthermore, the inconvenience of excessive information being output can be prevented.

[0114] As shown in Fig. 9, the information provider 13 may display information about off-display items both in stock in the back yard and out of stock in the back yard. By displaying information about off-display items even when the off-display items are out of stock in the back yard, the user can be notified of the products available at the store. This may result in a next sale.

[0115] Other configurations of the information processing device 10 of the fifth embodiment are similar to those of the information processing device 10 of the first to fourth embodiments.

[0116] According to the information processing device 10 of the fifth embodiment, the same effects as those of the information processing devices 10 of the first to fourth embodiments are realized.

[0117] Furthermore, the information processing device 10 of the fifth embodiment can output information indicating whether or not a product is in stock in the back yard. Such an information processing device 10 can prevent the inconvenience of losing sales opportunities due to delays in replenishing the display shelves despite there being stock in the back yard. Furthermore, by notifying the user using a distinctive method such as that shown in FIGS. 8 and 9, the user can easily grasp which products are out of stock on the display shelves but are still in stock in the back yard. Furthermore, by notifying the user using a distinctive method such as that shown in FIG. 9, the user can easily grasp which products are out of stock on the display shelves but are still available in the store.

[0118] <<Modifications>> Modifications applicable to all of the embodiments will be described below. In any of the modifications, the same effects as those of the above-described embodiment will be achieved.

[0119] <Variation 1> In the above embodiment, the user terminal functions as the information processing device 10. In Modification 1, a server functions as the information processing device 10. The server acquires various information from the user terminal. The various information includes at least one of an image generated by the user terminal, information input by the user to the user terminal, and current location information of the user terminal.

[0120] Then, the specifying unit 11, the acquiring unit 12, and the information providing unit 13 of the server (information processing device 10) execute the processes described in the above embodiment. The information providing unit 13 transmits the information to be output to the user terminal. The user terminal outputs the information received from the server using, for example, AR technology.

[0121] <Variation 2> In Variation 2, a scent space is set up in a store near a target product. When a user scans a code in the space with a user terminal, the user terminal displays a list of products. When a product is selected via the user terminal, the selection is sent to a scent emitting device set up in the scent space. The scent emitting device emits the scent of the selected product.

[0122] <Variation 3> In Modification 3, the information processing device 10 does not output information while the user (user terminal) is moving. The information processing device 10 outputs information when the user (user terminal) is not moving and is stationary. Whether the user terminal is moving or not can be determined based on the location information of the user terminal and an image generated by the user terminal.

[0123] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0124] In addition, in the flowcharts used in the above explanation, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0125] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. 1. An identification means for identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; an acquisition means for acquiring product information of the identified product; an information providing means for outputting comparison information for comparing the product information of the plurality of products when the product information of the plurality of products is acquired; An information processing device having the above. 2. The information processing device according to 1, wherein the information providing means outputs a screen displaying product information for a plurality of products as the comparison information. 3. The information providing means is: Using AR technology, information is displayed overlaid on the real world, 3. The information processing device according to 1 or 2, which changes the display mode of information to be displayed based on at least one of the type and amount of information to be displayed. 4. The identification means is Identify the product selected by the user, Identifying the user's current location within the store, Identifying a route from the user's current location to the display location of the product selected by the user; The information providing means 4. The information processing device according to any one of 1 to 3, which outputs information indicating the travel route. 5. The identification means is Based on the image of the display shelf, an out-of-stock product on the display shelf is identified; Identifying whether the non-displayed product is in stock in the backyard, The information providing means 5. The information processing device according to any one of 1 to 4, which outputs information indicating whether the stock of the undisplayed product is in the back room. 6. The identification means is Based on the image of the display shelf, an out-of-stock product on the display shelf is identified; The information providing means 6. The information processing device according to any one of 1 to 5, which uses AR technology to display information about the undisplayed product in a display area of ​​the undisplayed product on a display shelf. 7. The identification means is Identifying whether the non-displayed product is in stock in the back yard based on back yard inventory information indicating the inventory status in the back yard or an image taken of the back yard; The information providing means If there is stock in the backyard, information about the non-displayed product is displayed in the display area of ​​the non-displayed product on the display shelf; 7. The information processing device according to claim 6, wherein when there is no stock in the back yard, information about the undisplayed product is not displayed in the display area of ​​the undisplayed product on the display shelf. 8. The information providing means is: 8. The information processing device according to 6 or 7, wherein an image of the undisplayed product is displayed as information about the undisplayed product in a display area of ​​the display shelf where the undisplayed product is displayed. 9. One or more computers: Identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; Obtain product information for the identified product; An information processing method that, when the product information of a plurality of products is acquired, outputs comparison information that compares the product information of the plurality of products. 10. Computer an identification means for identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; an acquisition means for acquiring product information of the identified product; an information providing means for outputting, when the product information of a plurality of products is acquired, comparison information for comparing the product information of the plurality of products; A program that functions as a

[0126] Some or all of Supplements 2 to 8 that are dependent on the information processing device of Supplement 1 described above may also be dependent on the information processing method of Supplement 9 and the program of Supplement 10 in the same dependent relationship as Supplement 1 and Supplements 2 to 8. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems. [Explanation of symbols]

[0127] 10. Information processing equipment 11 Specific section 12 Acquisition Department 13 Information Provision Department 1A processor 2A Memory 3A input / output I / F 4A peripheral circuit 5A Bus

Claims

1. an identification means for identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; an acquisition means for acquiring product information of the identified product; an information providing means for outputting comparison information for comparing the product information of the plurality of products when the product information of the plurality of products is acquired; An information processing device having the above.

2. The information processing apparatus according to claim 1 , wherein the information providing means outputs a screen on which the product information of a plurality of products is arranged as the comparison information.

3. The information providing means Using AR (Augmented Reality) technology, information is displayed superimposed on the real world, 2. The information processing apparatus according to claim 1, wherein the display mode of the information to be displayed is changed based on at least one of the type and amount of the information to be displayed.

4. The identification means Identify the product selected by the user, Identifying the user's current location within the store, Identifying a route from the user's current location to the display location of the product selected by the user; The information providing means The information processing device according to claim 1 , wherein the information processing device outputs information indicating the travel route.

5. The identification means Based on the image of the display shelf, an out-of-stock product on the display shelf is identified; Identifying whether the non-displayed product is in stock in the backyard, The information providing means The information processing device according to claim 1 , wherein the information processing device outputs information indicating whether or not the non-displayed product is in stock in a back room.

6. The identification means Based on the image of the display shelf, an out-of-stock product on the display shelf is identified; The information providing means The information processing device according to claim 1 , wherein information about the undisplayed product is displayed in a display area of ​​the display shelf where the undisplayed product is displayed, using AR technology.

7. The identification means Identifying whether the non-displayed product is in stock in the back yard based on back yard inventory information indicating the inventory status in the back yard or an image taken of the back yard; The information providing means If there is stock in the backyard, information about the non-displayed product is displayed in the display area of ​​the non-displayed product on the display shelf; The information processing device according to claim 6, wherein when there is no stock in the backyard, information about the undisplayed product is not displayed in the display area of ​​the undisplayed product on the display shelf.

8. The information providing means 8. The information processing device according to claim 6, wherein an image of the undisplayed commodity is displayed as information about the undisplayed commodity in a display area of ​​the display shelf where the undisplayed commodity is displayed.

9. One or more computers Identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; Obtain product information for the identified product; An information processing method that, when the product information of a plurality of products is acquired, outputs comparison information that compares the product information of the plurality of products.

10. Computer, an identification means for identifying at least one of the products displayed in the store based on an image generated by a camera carried by a user moving around the store; an acquisition means for acquiring product information of the identified product; an information providing means for outputting, when the product information of a plurality of products is acquired, comparison information for comparing the product information of the plurality of products; A program that functions as a

Images